async, on-demand lazy rendering for jinja2 possible?

[est]

Hi all,

Been reading @igrigorik's posts https://www.igvita.com/archives/

I found that flush response as early as possible is crucial for a website
performance.

I wanna make Time To First Byte (TTTFB) happen as early as possible with a
Jinja2 site,

Suppose a typical Jinja2 template:


  {{ site_title }}


{% for item in items %}
  item
{% endfor %}



Let's presume site_title is static and known in view function from the
start, I wanna flush everything before  to the client as early as
possible, then items was a query from db, it's kinda slower, let's flush
the everything below   later in a second TCP packet.

This looks like impossible at the moment. I searched the Jinja2 docs, the
closest thing I've found is the streaming template API

jinja2.Template('Hello {{ name }}!').stream(name='foo')

Which still requires all variables to be ready before rendering.

Is there a way to stream partial template results with partial template
variables? This would be great help making faster websites. Other template
engine than Jinja2 is welcome as long as it's in python! Thanks in advance!
-- 
https://mail.python.org/mailman/listinfo/python-list

db agnostic admin for Flask?

[est]

Hi guys

Are there any db agnostic admin for Flask/Django?

Imagine you need CRUD lots of data from Mongodb, Redis, ini file or
even remote sources, if such admin allows aggregate data regardless of
DB types or syntax, it would be super useful.

Anything like this exists? or should I start build one from scratch?
-- 
http://mail.python.org/mailman/listinfo/python-list

SSE4a with ctypes in python? (gcc __builtin_popcount)

[est]

Hi guys,

Here is the sample code

http://stackoverflow.com/questions/6389841/efficiently-find-binary-strings-with-low-hamming-distance-in-large-set/6390606#6390606

static inline int distance(unsigned x, unsigned y)
{
return __builtin_popcount(x^y);
}

Is it possible to rewrite the above gcc code in python using ctypes
(preferably Win/*nix compatible)?

TIA!
-- 
http://mail.python.org/mailman/listinfo/python-list

SSE4a with ctypes in python? (gcc __builtin_popcount)

[est]

Hi guys,

Here is the sample code

http://stackoverflow.com/questions/6389841/efficiently-find-binary-strings-with-low-hamming-distance-in-large-set/6390606#6390606

static inline int distance(unsigned x, unsigned y)
{
return __builtin_popcount(x^y);
}

Is it possible to rewrite the above gcc code in python using ctypes
(preferably Win/*nix compatible)?

TIA!
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: minimal python27.dll?

[est]

On Apr 27, 11:15 pm, Tim Golden  wrote:
> Perhaps have a look at tinypy?
>
>    http://www.tinypy.org/
>
> Even if it's not exactly what you want, I expect that the
> author will have useful ideas / experience.
>
> TJG

Thanks, but I still need the completeness of CPython. AFAIK TinyPy is
a python implementation from scratch


On Apr 28, 4:06 am, "Martin v. Loewis"  wrote:
>
> OTOH, you'll also find that the CJK codecs use quite some space,
> if you don't need them, they give a rather impressive reduction.
> Likewise for the Unicode database, although you may actually need
> it in some cases.


On the CJK issue, why python ship its own codec, not using OS builtin?

If I don't need the full Unicode5.1 can I just map python's unicode
functions to some Win32 unicode API?
-- 
http://mail.python.org/mailman/listinfo/python-list

minimal python27.dll?

[est]

Hi guys,

I need to ship python runtime environment package on Windows, if I
want to stripping unnessasery functions from python27.dll to make it
as small as possible(and perhaps finally UPX it), which parts of
python27.dll do you think can be removed?

>From what I think, these parts are not needed when shipping with final
end-user product:

1. debugging
2. parse text .py files, because everything is already in bytecode

Any ideas? Critics?

Thanks in advance!
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Compiling xpython: No rule to make target `runathana.py', needed by `frozen/frozen.c'.

[est]

On Oct 10, 7:40 pm, est  wrote:
> I found an interesting project, it's a single 4.7 MB executable with
> stdlib+PIL+pdf r/w+magpy+wxpython environment
>
> But when compiling following the instructions
>
> http://www.xpython.org/#compiling
>
> there is an error
>
> make: *** No rule to make target `runathana.py', needed by `frozen/
> frozen.c'.  Stop.
>
> I am using Debian 4.0 32bit machine

Today I am trying to compile it using mingw (GNU Make 3.81) on Win7 32
bit and got this

In file included from Python/thread.c:121:
Python/thread_pthread.h: In function 'PyThread_start_new_thread':
Python/thread_pthread.h:200: error: aggregate value used where an
integer was expected
Python/thread_pthread.h: In function 'PyThread_get_thread_ident':
Python/thread_pthread.h:222: error: aggregate value used where an
integer was expected
make[1]: *** [Python/thread.o] Error 1
make[1]: Leaving directory `e:/dev/github.com/xpython/Python-linux'
mingw32-make: *** [Python-linux/python] Error 2

Can anyone help?
-- 
http://mail.python.org/mailman/listinfo/python-list

Compiling xpython: No rule to make target `runathana.py', needed by `frozen/frozen.c'.

[est]

I found an interesting project, it's a single 4.7 MB executable with
stdlib+PIL+pdf r/w+magpy+wxpython environment

But when compiling following the instructions

http://www.xpython.org/#compiling

there is an error

make: *** No rule to make target `runathana.py', needed by `frozen/
frozen.c'.  Stop.

I am using Debian 4.0 32bit machine
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: GUIs - A Modest Proposal

[est]

>
> Having said all that, I would like to eliminate some of the
> depedencie. At some point I'll probably re-do the Windows
> implementation using ctypes, because pywin32/mfc is hindering
> more than helping in some areas. I'm also thinking about ways
> to interface directly with Cocoa without going through pyobjc.
> But all that is some way off in the future after I get the API
> nailed down more.
>
> --
> Greg


that would be awesome!
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: What's the largest python/django powered website in the world?

[est]

On May 31, 8:14 am, Benjamin Kaplan  wrote:
> On Sun, May 30, 2010 at 4:27 PM, est  wrote:
>
> > > I'm afraid you'll need to define what you mean by "python powered".
>
> > Except database, presentation layer, major business logic is done by
> > python.
>
> > Except Google/youtube, what's next?
> > --
> >http://mail.python.org/mailman/listinfo/python-list
>
> http://www.reddit.comhttp://github.com/reddit/reddit
>
> I believe they recently said that they're at 7.5 million users and 270
> million page views per month. And the repo is 71% Python with most of
> the rest as Javascript.

http://www.alexa.com/siteinfo/douban.com

is written in quixote, python 2.6
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: What's the largest python/django powered website in the world?

[est]

> I'm afraid you'll need to define what you mean by "python powered".

Except database, presentation layer, major business logic is done by
python.

Except Google/youtube, what's next?
-- 
http://mail.python.org/mailman/listinfo/python-list

What's the largest python/django powered website in the world?

[est]

Hi list,

just curious, what's the largest python powered website in the world?

and what's the largest django powered website in the world? Is it
disqus.com?
-- 
http://mail.python.org/mailman/listinfo/python-list

modify a exe's RT_ACCELERATOR table using pefile module

[est]

Hi list,

I need to modify a exe's RT_ACCELERATOR table, rename F1 to Ctrl+F1,
and assign hotkey F1 to another function

Is there a way to do it with pefile/python? A sample code is
appreciated. Thanks!
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: ANN: psyco V2

[est]

On Jul 17, 10:48 am, Christian Tismer  wrote:
> Announcing Psyco V2 source release
> --
>
> This is the long awaited announcement of Psyco V2.
>
> Psyco V2 is a continuation of the well-known psyco project,
> which was called finished and was dis-continued by its author
> Armin Rigo in 2005, in favor of the PyPy project.
>
> This is a new project, using Psyco's code base with permission
> of Armin. Questions and complaints should go to me
> (tis...@stackless.com) or the mailing list
> (psyco-de...@lists.sourceforge.net);
> Armin is explicitly not in charge of (t)his project any longer!
>
> As one of the founders and an active member of the PyPy
> project, I was very happy to be invited to work on Psyco
> V2, by FATTOC, LLC. Psyco V2 tries to extend on the original Psyco
> approach "an extension module that just makes Python faster".
>
> Psyco is a just-in-time compiler that accelerates arbitrary
> Python code by specialization. We believe that Psyco's approach
> can be carried out much further than it was tried so far, when
> it's first version was abandoned.
>
> This first V2 release is source-only. There is no web-site, yet,
> and there are no binaries for download. These will be available
> in a few days onhttp://www.psyco.org.
>
> For the time being, please stick with subversion access,
> building the extension module from source code. The repository
> is here:
>
>      http://codespeak.net/svn/psyco/v2/dist
>
> Check-out the repository, and run the setup.py script,
> given that you have access to a C compiler.
>
> Psyco V2 will run on X86 based 32 bit Linux, 32 bit Windows,
> and Mac OS X. Psyco is not supporting 64 bit, yet. But it
> is well being considered.
>
> The current improvements are, shortly:
>
>    - Support for Python 2.4, 2.5 and 2.6
>    - a lot of new builtins
>    - generators, fast and fully supported.
>
> More information is coming soon onhttp://www.psyco.org.
>
> This is the beginning of a series of new Psyco versions.
> Many more improvements are prepared and about to be published,
> soon, starting with the current version 2.0.0 .
>
> Stay tuned, this is just the beginning of psyco's re-birth!
>
> For questions about Psyco V2, please join the mailing list
>
>      psyco-de...@lists.sourceforge.net
>
> or contact me on IRC:
>
>      #psyco on irc.freenode.net .
>
> Psyco V2 is fundamentally supported by FATTOC, LLC.
> Seehttp://www.fattoc.com.
>
> Without their continuous support, this work would not have
> been possible at all. I wish to express my deepest thanks
> to FATTOC, for allowing me to continue on Psyco with all the
> energy that this ambitious project needs, and will need.
>
> Further special thanks are going to
> Armin Rigo, John Benediktsson, David Salomon, Miki Tebeka,
> Raymond Hettinger, Fabrizio Milo, Michael Foord,
> Dinu Gherman, Stephan Diehl, Laura Creighton and Andrea Tismer,
> for all the support and discussions.
>
> Looking forward to a great future of Psyco!
>
> July 17, 2009
> --
> Christian Tismer             :^)   
> tismerysoft GmbH             :     Have a break! Take a ride on Python's
> Johannes-Niemeyer-Weg 9A     :    *Starship*http://starship.python.net/
> 14109 Berlin                 :     PGP key ->http://wwwkeys.pgp.net/
> work +49 30 802 86 56  mobile +49 173 24 18 776  fax +49 30 80 90 57 05
> PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
>        whom do you want to sponsor today?  http://www.stackless.com/

congrats!

btw is psyco.org owned?
-- 
http://mail.python.org/mailman/listinfo/python-list

ctypes to customize MSN now playing status

[est]

Hi guys, I am trying to switch MSN/WLM now playing status using ctypes
with python, here's my code

import ctypes, win32con, win32api, win32gui

FindWindow = ctypes.windll.User32.FindWindowA
SendMessage = ctypes.windll.User32.SendMessageA

hWnd = FindWindow('MsnMsgrUIManager', None)

s = u"\\0Music\\01\\0{test}\\0\\0\\0\\0\\0"

class MsnData(ctypes.Structure):
_fields_ = [("dwData", ctypes.c_int),
("cbData", ctypes.c_int),
("lpData", ctypes.c_wchar_p),
   ]

msndata = MsnData(0x547, 256, ctypes.c_wchar_p(s))

SendMessage(hWnd, win32con.WM_COPYDATA, 0, ctypes.byref(msndata))

The code is not working, could anybody help?
--
http://mail.python.org/mailman/listinfo/python-list

Where can I get Lucid Toolkit?

[est]

Hi guys,

I tried to grab source code for  Lucid Toolkit 
http://www.clarographics.org/svn_details
which was mentioned http://wiki.python.org/moin/GuiProgramming

So is this project abandoned? Is there any backup that I can get the
code?

Any other recommands for simple GUI development, with out large
library runtime(qt, gtk, wx) ?
--
http://mail.python.org/mailman/listinfo/python-list

Re: a question about Chinese characters in a Python Program

[est]

On Oct 20, 11:46 pm, Steven D'Aprano <[EMAIL PROTECTED]
cybersource.com.au> wrote:
> On Mon, 20 Oct 2008 06:30:09 -0700, est wrote:
> > Like I said, str() should NOT throw an exception BY DESIGN, it's a basic
> > language standard.
>
> int() is also a basic language standard, but it is perfectly acceptable
> for int() to raise an exception if you ask it to convert something into
> an integer that can't be converted:
>
> int("cat")
>
> What else would you expect int() to do but raise an exception?
>
> If you ask str() to convert something into a string which can't be
> converted, then what else should it do other than raise an exception?
> Whatever answer you give, somebody else will argue it should do another
> thing. Maybe I want failed characters replaced with '?'. Maybe Fred wants
> failed characters deleted altogether. Susan wants UTF-16. George wants
> Latin-1.
>
> The simple fact is that there is no 1:1 mapping from all 65,000+ Unicode
> characters to the 256 bytes used by byte strings, so there *must* be an
> encoding, otherwise you don't know which characters map to which bytes.
>
> ASCII has the advantage of being the lowest common denominator. Perhaps
> it doesn't make too many people very happy, but it makes everyone equally
> unhappy.
>
> > str() is not only a convert to string function, but
> > also a serialization in most cases.(e.g. socket) My simple suggestion
> > is: If it's a unicode character, output as UTF-8;
>
> Why UTF-8? That will never do. I want it output as UCS-4.
>
> > other wise just ouput
> > byte array, please do not encode it with really stupid range(128) ASCII.
> > It's not guessing, it's totally wrong.
>
> If you start with a byte string, you can always get a byte string:
>
> >>> s = '\x96 \xa0 \xaa'  # not ASCII characters
> >>> s
> '\x96 \xa0 \xaa'
> >>> str(s)
>
> '\x96 \xa0 \xaa'
>
> --
> Steven

In fact Python handles characters well than most other open-source
programming languages. But still:

1. You can explain str() in 1000 ways, there are 1001 more confusing
error on all kinds of python apps. (Not only some of the scripts I've
written, but also famous enough apps like Boa Constructor
http://i36.tinypic.com/1gqekh.jpg. This sucks hard, right?)


2. Anyone please kindly tell me how can I define a customized encoding
(namely 'ansi') which handles range(256) so I can
sys.setdefaultencoding('ansi') once and for all?
--
http://mail.python.org/mailman/listinfo/python-list

Re: a question about Chinese characters in a Python Program

[est]

On Oct 20, 6:47 pm, Paul Boddie <[EMAIL PROTECTED]> wrote:
> On 20 Okt, 07:32, est <[EMAIL PROTECTED]> wrote:
>
>
>
> > Personally I call it a serious bug in python
>
> Normally I'd entertain the possibility of bugs in Python, but your
> reasoning is a bit thin (inhttp://bugs.python.org/issue3648):"Why
> cann't Python just define ascii to range(256)"
>
> I do accept that it can be awkward to output text to the console, for
> example, but you have to consider that the console might not be
> configured to display any character you can throw at it. My console is
> configured for ISO-8859-15 (something like your magical "ascii to
> range(256)" only where someone has to decide what those 256 characters
> actually are), but that isn't going to help me display CJK characters.
> A solution might be to generate UTF-8 and then get the user to display
> the output in an appropriately configured application, but even then
> someone has to say that it's UTF-8 and not some other encoding that's
> being used. As discussed in another recent thread, Python 2.x does
> make some reasonable guesses about such matters to the extent that
> it's possible automatically (without magical knowledge).
>
> There is also the problem about use of the "str" built-in function or
> any operation where some Unicode object may be converted to a plain
> string. It is now recommended that you only convert to plain strings
> when you need to produce a sequence of bytes (for output, for
> example), and that you indicate how the Unicode values are encoded as
> bytes (by specifying an encoding). Python 3.x doesn't really change
> this: it just makes the Unicode/text vs. bytes distinction more
> obvious.
>
> Paul

Thanks for the long comment Paul, but it didn't help massive errors in
Python encoding.

IMHO it's even better to output wrong encodings rather than halt the
WHOLE damn program by an exception

When debugging encoding problems, the solution is simple. If
characters display wrong, switch to another encoding, one of them must
be right.

But it's tiring in python to deal with encodings, you have to wrap
EVERY SINGLE character expression with try ... except ... just imagine
what pain it is.

Just like the example I gave in Google Groups, u'\ue863' can NEVER be
encoded into '\xfe\x9f'. Not a chance, because python REFUSE to handle
a byte that is greater than range(128).

Strangely the 'mbcs' encoding system can. Does 'mbcs' have magic or
something? But it's Windows-specific

Dealing with character encodings is really simple. AFAIK early
encoding before Unicode, although they have many names, are all based
on hacks. Take Chinese characters as an example. They are called
GB2312 encoding, in fact it is totally compatible with range(256)
ANSI. (There are minor issues like display half of a wide-character in
a question mark ? but at least it's readable) If you just output
serials of byte array, it IS GB2312. The same is true with BIG5, JIS,
etc.


Like I said, str() should NOT throw an exception BY DESIGN, it's a
basic language standard. str() is not only a convert to string
function, but also a serialization in most cases.(e.g. socket) My
simple suggestion is: If it's a unicode character, output as UTF-8;
other wise just ouput byte array, please do not encode it with really
stupid range(128) ASCII. It's not guessing, it's totally wrong.
--
http://mail.python.org/mailman/listinfo/python-list

Re: a question about Chinese characters in a Python Program

[est]

On Oct 20, 10:48 am, Liang Chen <[EMAIL PROTECTED]> wrote:
> Hope you all had a nice weekend.
>
> I have a question that I hope someone can help me out. I want to run a Python 
> program that uses Tkinter for the user interface (GUI). The program allows me 
> to type Chinese characters, but neverthelss is unable to show them up on 
> screen. The follow is some of the error message I received after I logged off 
> the program:
>
> "Could not write output: , 'ascii' 
> codec can't encode characters in position 0-1: ordinal not in range (128)"
>
> Any suggestion will be appreciated.
>
> Sincerely,
>
> Liang
>
> Liang Chen,Ph.D.
> Assistant Professor
> University of Georgia
> Communication Sciences and Special Education
> 542 Aderhold Hall
> Athens, GA 30602
>
> Phone: 706-542-4566

Personally I call it a serious bug in python, but sadly most of python
community members do not agree
. It may be a internal str() that caused this issue.

https://groups.google.com/group/comp.lang.python/t/ca6ade6b6f5f3052
http://bugs.python.org/issue3648

--
http://mail.python.org/mailman/listinfo/python-list

Re: TypeError: can't pickle HASH objects?

[est]

On Oct 2, 5:07 pm, "Gabriel Genellina" <[EMAIL PROTECTED]> wrote:
> En Wed, 01 Oct 2008 16:50:05 -0300, est <[EMAIL PROTECTED]> escribió:
>
>
>
>
>
> >>>> import md5
> >>>> a=md5.md5()
> >>>> import pickle
> >>>> pickle.dumps(a)
> > Traceback (most recent call last):
> >   File "", line 1, in 
> >   File "C:\Python25\lib\pickle.py", line 1366, in dumps
> >     Pickler(file, protocol).dump(obj)
> >   File "C:\Python25\lib\pickle.py", line 224, in dump
> >     self.save(obj)
> >   File "C:\Python25\lib\pickle.py", line 306, in save
> >     rv = reduce(self.proto)
> >   File "C:\Python25\lib\copy_reg.py", line 69, in _reduce_ex
> >     raise TypeError, "can't pickle %s objects" % base.__name__
> > TypeError: can't pickle HASH objects
>
> > Why can't I pickle a md5 object? Is it because md5 algorithm needs to
> > read 512-bits at a time?
>
> > I need to md5() some stream, pause(python.exe quits), and resume
> > later.  It seems that the md5 and hashlib in  std module could not be
> > serialized?
>
> Yep, they're implemented in C and have no provision for serializing.
> If you can use the old _md5 module, it is far simpler to serialize; a  
> md5object just contains a small struct with 6 integers and 64 chars, no  
> pointers.
>
> With some help from ctypes (and a lot of black magic!) one can extract the  
> desired state, and restore it afterwards:
>
> --- begin code ---
> import _md5
> import ctypes
>
> assert _md5.MD5Type.__basicsize__==96
>
> def get_md5_state(m):
>      if type(m) is not _md5.MD5Type:
>          raise TypeError, 'not a _md5.MD5Type instance'
>      return ctypes.string_at(id(m)+8, 88)
>
> def set_md5_state(m, state):
>      if type(m) is not _md5.MD5Type:
>          raise TypeError, 'not a _md5.MD5Type instance'
>      if not isinstance(state,str):
>          raise TypeError, 'state must be str'
>      if len(state)!=88:
>          raise ValueError, 'len(state) must be 88'
>      a88 = ctypes.c_char*88
>      pstate = a88(*list(state))
>      ctypes.memmove(id(m)+8, ctypes.byref(pstate), 88)
>
> --- end code ---
>
> py> m1 = _md5.new()
> py> m1.update("this is a ")
> py> s = get_md5_state(m1)
> py> del m1
> py>
> py> m2 = _md5.new()
> py> set_md5_state(m2, s)
> py> m2.update("short test")
> py> print m2.hexdigest()
> 95ad1986e9a9f19615cea00b7a44b912
> py> print _md5.new("this is a short test").hexdigest()
> 95ad1986e9a9f19615cea00b7a44b912
>
> The code above was only tested with Python 2.5.2 on Windows, not more than  
> you can see. It might or might not work with other versions or platforms.  
> It may even create a (small) black hole and eat your whole town. Use at  
> your own risk.
>
> --
> Gabriel Genellina- Hide quoted text -
>
> - Show quoted text -

WOW! I never expected python could be coded like that! Thanks a lot!


On Oct 2, 5:19 pm, "Aaron \"Castironpi\" Brady" <[EMAIL PROTECTED]>
wrote:
> On Oct 2, 4:03 am, est <[EMAIL PROTECTED]> wrote:
>
>
>
>
>
> > On Oct 2, 4:22 pm, "Aaron \"Castironpi\" Brady" <[EMAIL PROTECTED]>
> > wrote:
>
> > > On Oct 2, 2:44 am, est <[EMAIL PROTECTED]> wrote:
>
> > > > On Oct 2, 1:51 pm, "James Mills" <[EMAIL PROTECTED]> wrote:
>
> > > > > On Thu, Oct 2, 2008 at 3:34 PM, est <[EMAIL PROTECTED]> wrote:
> > > > > > wow. It's giga-size file. I need stream reading it, md5 it. It may
> > > > > > break for a while.
>
> > > > > So use generators and consume the stream ?
>
> > > > > --JamesMills
>
> > > > > --
> > > > > --
> > > > > -- "Problems are solved by method"
>
> > > > no, I need to serialize half-finished digest, not file stream.
>
> > > > Anyone got solution?
>
> > > I am looking at '_hashopenssl.c'.  If you can find the implementation
> > > of EVP_DigestUpdate, I'll give it a shot to help you write a ctypes
> > > hack to store and write its state.- Hide quoted text -
>
> > > - Show quoted text -
>
> >http://cvs.openssl.org/fileview?f=openssl/crypto/evp/digest.c
>
> > int EVP_DigestUpdate(EVP_MD_CTX *ctx, const void *data,
> >  size_t count)
> > {
> > #ifdef OPENSSL_FIPS
> > FIPS_selftest_check();
> > #endif
> > return ctx->digest->update(ctx,data,count);
> > }
>
> > is this one?
>
> Oops, I needed 'EVP_MD_CTX'.  I went Googling and found it.
>
> http://www.google.com/codesearch?hl=en&q=struct+EVP_MD_CTX+show:mV3VB...
>
> But does Gabriel's work for you?- Hide quoted text -
>
> - Show quoted text -

Still need some hack with py2.5 on linux. Maybe I just need soft link /
usr/lib/python2.4/lib-dynload/md5.so to py2.5 :-)
py2.4 is pre-installed on most of the servers, I think.
--
http://mail.python.org/mailman/listinfo/python-list

Re: TypeError: can't pickle HASH objects?

[est]

On Oct 2, 4:22 pm, "Aaron \"Castironpi\" Brady" <[EMAIL PROTECTED]>
wrote:
> On Oct 2, 2:44 am, est <[EMAIL PROTECTED]> wrote:
>
>
>
>
>
> > On Oct 2, 1:51 pm, "James Mills" <[EMAIL PROTECTED]> wrote:
>
> > > On Thu, Oct 2, 2008 at 3:34 PM, est <[EMAIL PROTECTED]> wrote:
> > > > wow. It's giga-size file. I need stream reading it, md5 it. It may
> > > > break for a while.
>
> > > So use generators and consume the stream ?
>
> > > --JamesMills
>
> > > --
> > > --
> > > -- "Problems are solved by method"
>
> > no, I need to serialize half-finished digest, not file stream.
>
> > Anyone got solution?
>
> I am looking at '_hashopenssl.c'.  If you can find the implementation
> of EVP_DigestUpdate, I'll give it a shot to help you write a ctypes
> hack to store and write its state.- Hide quoted text -
>
> - Show quoted text -


http://cvs.openssl.org/fileview?f=openssl/crypto/evp/digest.c

int EVP_DigestUpdate(EVP_MD_CTX *ctx, const void *data,
 size_t count)
{
#ifdef OPENSSL_FIPS
FIPS_selftest_check();
#endif
return ctx->digest->update(ctx,data,count);
}


is this one?
--
http://mail.python.org/mailman/listinfo/python-list

Re: TypeError: can't pickle HASH objects?

[est]

On Oct 2, 1:51 pm, "James Mills" <[EMAIL PROTECTED]> wrote:
> On Thu, Oct 2, 2008 at 3:34 PM, est <[EMAIL PROTECTED]> wrote:
> > wow. It's giga-size file. I need stream reading it, md5 it. It may
> > break for a while.
>
> So use generators and consume the stream ?
>
> --JamesMills
>
> --
> --
> -- "Problems are solved by method"

no, I need to serialize half-finished digest, not file stream.

Anyone got solution?
--
http://mail.python.org/mailman/listinfo/python-list

Re: TypeError: can't pickle HASH objects?

[est]

On Oct 2, 11:27 am, "Aaron \"Castironpi\" Brady"
<[EMAIL PROTECTED]> wrote:
> On Oct 1, 2:50 pm, est <[EMAIL PROTECTED]> wrote:
>
>
>
>
>
> > >>> import md5
> > >>> a=md5.md5()
> > >>> import pickle
> > >>> pickle.dumps(a)
>
> > Traceback (most recent call last):
> >   File "", line 1, in 
> >   File "C:\Python25\lib\pickle.py", line 1366, in dumps
> >     Pickler(file, protocol).dump(obj)
> >   File "C:\Python25\lib\pickle.py", line 224, in dump
> >     self.save(obj)
> >   File "C:\Python25\lib\pickle.py", line 306, in save
> >     rv = reduce(self.proto)
> >   File "C:\Python25\lib\copy_reg.py", line 69, in _reduce_ex
> >     raise TypeError, "can't pickle %s objects" % base.__name__
> > TypeError: can't pickle HASH objects
>
> > Why can't I pickle a md5 object? Is it because md5 algorithm needs to
> > read 512-bits at a time?
>
> > I need to md5() some stream, pause(python.exe quits), and resume
> > later.  It seems that the md5 and hashlib in  std module could not be
> > serialized?
>
> > Do I have to implement md5 algorithm again for this special occasion?
>
> > Or is there anyway to assige a digest when creating md5 objects?
>
> Can you just pickle the stream, the part of it you've read so far?- Hide 
> quoted text -
>
> - Show quoted text -

wow. It's giga-size file. I need stream reading it, md5 it. It may
break for a while.
--
http://mail.python.org/mailman/listinfo/python-list

TypeError: can't pickle HASH objects?

[est]

>>> import md5
>>> a=md5.md5()
>>> import pickle
>>> pickle.dumps(a)
Traceback (most recent call last):
  File "", line 1, in 
  File "C:\Python25\lib\pickle.py", line 1366, in dumps
Pickler(file, protocol).dump(obj)
  File "C:\Python25\lib\pickle.py", line 224, in dump
self.save(obj)
  File "C:\Python25\lib\pickle.py", line 306, in save
rv = reduce(self.proto)
  File "C:\Python25\lib\copy_reg.py", line 69, in _reduce_ex
raise TypeError, "can't pickle %s objects" % base.__name__
TypeError: can't pickle HASH objects

Why can't I pickle a md5 object? Is it because md5 algorithm needs to
read 512-bits at a time?

I need to md5() some stream, pause(python.exe quits), and resume
later.  It seems that the md5 and hashlib in  std module could not be
serialized?

Do I have to implement md5 algorithm again for this special occasion?

Or is there anyway to assige a digest when creating md5 objects?
--
http://mail.python.org/mailman/listinfo/python-list

Re: str() should convert ANY object to a string without EXCEPTIONS !

[est]

On Sep 28, 7:12 pm, Lie <[EMAIL PROTECTED]> wrote:
> On Sep 28, 3:35 pm, est <[EMAIL PROTECTED]> wrote:
>
> > > Because that's how ASCII is defined.
> > > Because that's how ASCII is defined.  ASCII is a 7-bit code.
>
> > Then why can't python use another default encoding internally
> > range(256)?
>
> > > Python refuses to guess and tries the lowest common denominator -- ASCII 
> > > -- instead.
>
> > That's the problem. ASCII is INCOMPLETE!
>
> What do you propose? Use mbsc and smack out linux computers? Use KOI
> and make non-Russians suicide? Use GB and shot dead non-Chinese? Use
> latin-1 and make emails servers scream?
>
> > If Python choose another default encoding which handles range(256),
> > 80% of python unicode encoding problems are gone.
>
> > It's not HARD to process unicode, it's just python & python community
> > refuse to correct it.
>
> Python's unicode support is already correct. Only your brainwave have
> not been tuned to it yet.
>
> > > stop dreaming of a magic solution
>
> > It's not 'magic' it's a BUG. Just print 0x7F to 0xFF to console,
> > what's wrong
>
> > > Isn't that more or less the same as telling the OP to use unicode() 
> > > instead of str()?
>
> > sockets could handle str() only. If you throw unicode objects to a
> > socket, it will automatically call str() and cause an error.

Have you ever programmed with CJK characters before?
--
http://mail.python.org/mailman/listinfo/python-list

Re: str() should convert ANY object to a string without EXCEPTIONS !

[est]

On Sep 28, 6:15 pm, Marc 'BlackJack' Rintsch <[EMAIL PROTECTED]> wrote:
> On Sun, 28 Sep 2008 01:35:11 -0700, est wrote:
> >> Because that's how ASCII is defined.
> >> Because that's how ASCII is defined.  ASCII is a 7-bit code.
>
> > Then why can't python use another default encoding internally
> > range(256)?
>
> Because that doesn't suffice.  Unicode code points can be >255.
>
> > If Python choose another default encoding which handles range(256), 80%
> > of python unicode encoding problems are gone.
>
> 80% of *your* problems with it *seems* to be gone then.
>
> > It's not HARD to process unicode, it's just python & python community
> > refuse to correct it.
>
> It is somewhat hard to deal with unicode because many don't want to think
> about it or don't grasp the relationship between encodings, byte values,
> and characters.  Including you.
>
> >> stop dreaming of a magic solution
>
> > It's not 'magic' it's a BUG. Just print 0x7F to 0xFF to console, what's
> > wrong
>
> What do you mean by "just print 0x7F to 0xFF"?  For example if I have ``s
> = u'Smørebrød™'`` what bytes should ``str(s)`` produce and why those and
> not others?
>
> >> Isn't that more or less the same as telling the OP to use unicode()
> >> instead of str()?
>
> > sockets could handle str() only. If you throw unicode objects to a
> > socket, it will automatically call str() and cause an error.
>
> Because *you* have to tell explicitly how the unicode object should be
> encoded as bytes.  Python can't do this automatically because it has *no
> idea* what the process at the other end of the socket expects.
>
> Now you are complaining that Python chooses ASCII.  If it is changed to
> something else, like MBCS, others start complaining why it is MBCS and
> not something different.  See: No fix, just moving the problem to someone
> else.
>
> Ciao,
>         Marc 'BlackJack' Rintsch

Well, you succeseded in putting all blame to myself alone. Great.

When you guy's are dealing with CJK characters in the future, you'll
find out what I mean.

In fact Boa Constructor keeps prompting ASCII and range(128) error on
my Windows. That's pretty cool.
--
http://mail.python.org/mailman/listinfo/python-list

Re: str() should convert ANY object to a string without EXCEPTIONS !

[est]

On Sep 28, 4:38 pm, Steven D'Aprano <[EMAIL PROTECTED]
cybersource.com.au> wrote:
> On Sat, 27 Sep 2008 22:37:09 -0700, est wrote:
> >>>> str(u'\ue863')
> > Traceback (most recent call last):
> >   File "", line 1, in 
> > UnicodeEncodeError: 'ascii' codec can't encode character u'\ue863' in
> > position 0
> > : ordinal not in range(128)
>
> > FAIL.
>
> What result did you expect?
>
> [...]
>
> > The problem is, why the f**k set ASCII encoding to range(128) 
> > while str() is internally byte array it should be handled in range(256)
> > !!
>
> To quote Terry Pratchett:
>
>     "What sort of person," said Salzella patiently, "sits down and
>     *writes* a maniacal laugh? And all those exclamation marks, you
>     notice? Five? A sure sign of someone who wears his underpants
>     on his head." -- (Terry Pratchett, Maskerade)
>
> In any case, even if the ASCII encoding used all 256 possible bytes, you
> still have a problem. Your unicode string is a single character with
> ordinal value 59491:
>
> >>> ord(u'\ue863')
>
> 59491
>
> You can't fit 59491 (or more) characters into 256, so obviously some
> unicode chars aren't going to fit into ASCII without some sort of
> encoding. You show that yourself:
>
> u'\ue863'.encode('mbcs')  # Windows only
>
> But of course 'mbcs' is only one possible encoding. There are others.
> Python refuses to guess which encoding you want. Here's another:
>
> u'\ue863'.encode('utf-8')
>
> --
> Steven

OK, I am tired of arguing these things since python 3.0 fixed it
somehow.

Can anyone tell me how to customize a default encoding, let's say
'ansi' which handles range(256) ?
--
http://mail.python.org/mailman/listinfo/python-list

Re: str() should convert ANY object to a string without EXCEPTIONS !

[est]

> Because that's how ASCII is defined.
> Because that's how ASCII is defined.  ASCII is a 7-bit code.

Then why can't python use another default encoding internally
range(256)?

> Python refuses to guess and tries the lowest common denominator -- ASCII -- 
> instead.

That's the problem. ASCII is INCOMPLETE!

If Python choose another default encoding which handles range(256),
80% of python unicode encoding problems are gone.

It's not HARD to process unicode, it's just python & python community
refuse to correct it.

> stop dreaming of a magic solution

It's not 'magic' it's a BUG. Just print 0x7F to 0xFF to console,
what's wrong

> Isn't that more or less the same as telling the OP to use unicode() instead 
> of str()?

sockets could handle str() only. If you throw unicode objects to a
socket, it will automatically call str() and cause an error.
--
http://mail.python.org/mailman/listinfo/python-list

str() should convert ANY object to a string without EXCEPTIONS !

[est]

>From python manual

str( [object])

Return a string containing a nicely printable representation of an
object. For strings, this returns the string itself. The difference
with repr(object) is that str(object) does not always attempt to
return a string that is acceptable to eval(); its goal is to return a
printable string. If no argument is given, returns the empty string,
''.


now we try this under windows:

>>> str(u'\ue863')
Traceback (most recent call last):
  File "", line 1, in 
UnicodeEncodeError: 'ascii' codec can't encode character u'\ue863' in
position 0
: ordinal not in range(128)

FAIL.

also almighty Linux

Python 2.3.4 (#1, Feb  6 2006, 10:38:46)
[GCC 3.4.5 20051201 (Red Hat 3.4.5-2)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> str(u'\ue863')
Traceback (most recent call last):
  File "", line 1, in ?
UnicodeEncodeError: 'ascii' codec can't encode character u'\ue863' in
position 0: ordinal not in range(128)

Python 2.4.4 (#2, Apr  5 2007, 20:11:18)
[GCC 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> str(u'\ue863')
Traceback (most recent call last):
  File "", line 1, in ?
UnicodeEncodeError: 'ascii' codec can't encode character u'\ue863' in
position 0: ordinal not in range(128)

Python 2.5 (release25-maint, Jul 20 2008, 20:47:25)
[GCC 4.1.2 20061115 (prerelease) (Debian 4.1.1-21)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> str(u'\ue863')
Traceback (most recent call last):
  File "", line 1, in 
UnicodeEncodeError: 'ascii' codec can't encode character u'\ue863' in
position 0: ordinal not in range(128)


The problem is, why the f**k set ASCII encoding to range(128) 
while str() is internally byte array it should be handled in
range(256) !!

http://bugs.python.org/issue3648

One possible solution(Windows Only)

>>> str(u'\ue863'.encode('mbcs'))
'\xfe\x9f'
>>> print u'\ue863'.encode('mbcs')
��


I now spending 60% of my developing time dealing with ASCII range(128)
errors. It was PAIN!!

Please fix this issue.

http://bugs.python.org/issue3648

Please.
--
http://mail.python.org/mailman/listinfo/python-list

Re: forkpty not working?

[est]

On Mar 21, 2:58 am, Dan Stromberg <[EMAIL PROTECTED]> wrote:
> If you google a bit, I believe you'll find one or more python modules for
> working with ssh.
>
> Also, if you want to roll your own, the easiest way to get around the
> password requirement is to use ssh passwordless authentication using DSA
> or RSA public/private keypairs:
>
> http://stromberg.dnsalias.org/~dstromberg/ssh-keys.html
>
> As far as why the code below doesn't open your "it_worked" file - exec
> replaces the current process with the specified process - so the current
> process effectively goes away on exec.
>
> On Thu, 20 Mar 2008 00:40:49 -0700, est wrote:
> > Hello everyone. I am trying to write a bash/ssh wrapper in python so
> > python scripts could interact with bash/ssh.
>
> > Because when input passwords to ssh it requires a tty rather than stdin
> > pipe, so i have to use a pty module to do that.
>
> > I copied this snippet from this thread
> >http://groups.google.com/group/comp.lang.python/browse_thread/
>
> thread/6bbc3d36b4e6ff55/
>
>
>
>
>
> > def rcmd(user, rhost, pw, cmd):
> >         #Fork a child process, using a new pseudo-terminal as the
> > child's controlling terminal.
> >         pid, fd = os.forkpty()
> >         # If Child; execute external process
> >         if pid == 0:
> >                 os.execv("/bin/ssh", ["/bin/ssh", "-l", user, rhost] +
> > cmd)
> >                 x=open("it_worked.txt", "w") #output a file for test
> >                 x.write("xxx")
> >                 x.close()
> >         #if parent, read/write with child through file descriptor else:
> >                 pause()
> >                 #Get password prompt; ignore
> >                 os.read(fd, 1000)
> >                 pause()
> >                 #write password
> >                 os.write(fd, pw + "\n")
> >                 pause()
> >                 res = ''
> >                 #read response from child process
> >                 s = os.read(fd,1 )
> >                 while s:
> >                         res += s
> >                         s = os.read(fd, 1)
> >                 return res
>
> > As far as I can see the code is not working, when called the function
> > rcmd() there is no file it_worked.txt spawned in the directory. I am
> > n00b to POSIX, so anyone please give me some hint? what happened when
> > os.forkpty()?- Hide quoted text -
>
> - Show quoted text -

Thanks for the reply! What's your recommandation for a bash wrapper? I
am trying to implement some simple stuff which functions like ipython.
-- 
http://mail.python.org/mailman/listinfo/python-list

forkpty not working?

[est]

Hello everyone. I am trying to write a bash/ssh wrapper in python so
python scripts could interact with bash/ssh.

Because when input passwords to ssh it requires a tty rather than
stdin pipe, so i have to use a pty module to do that.

I copied this snippet from this thread
http://groups.google.com/group/comp.lang.python/browse_thread/thread/6bbc3d36b4e6ff55/

def rcmd(user, rhost, pw, cmd):
#Fork a child process, using a new pseudo-terminal as the
child's controlling terminal.
pid, fd = os.forkpty()
# If Child; execute external process
if pid == 0:
os.execv("/bin/ssh", ["/bin/ssh", "-l", user, rhost] +
cmd)
x=open("it_worked.txt", "w") #output a file for test
x.write("xxx")
x.close()
#if parent, read/write with child through file descriptor
else:
pause()
#Get password prompt; ignore
os.read(fd, 1000)
pause()
#write password
os.write(fd, pw + "\n")
pause()
res = ''
#read response from child process
s = os.read(fd,1 )
while s:
res += s
s = os.read(fd, 1)
return res

As far as I can see the code is not working, when called the function
rcmd() there is no file it_worked.txt spawned in the directory. I am
n00b to POSIX, so anyone please give me some hint? what happened when
os.forkpty()?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: n00b with urllib2: How to make it handle cookie automatically?

[est]

On Feb 25, 5:46 am, 7stud <[EMAIL PROTECTED]> wrote:
> On Feb 24, 4:41 am, est <[EMAIL PROTECTED]> wrote:
>
>
>
>
>
> > On Feb 23, 2:42 am, Rob Wolfe <[EMAIL PROTECTED]> wrote:
>
> > > est <[EMAIL PROTECTED]> writes:
> > > > Hi all,
>
> > > > I need urllib2 do perform series of HTTP requests with cookie from
> > > > PREVIOUS request(like our browsers usually do ). Many people suggest I
> > > > use some library(e.g. pycURL) instead but I guess it's good practise
> > > > for a python beginner to DIY something rather than use existing tools.
>
> > > > So my problem is how to expand the urllib2 class
>
> > > > from cookielib import CookieJar
> > > > class SmartRequest():
> > > >     cj=CookieJar()
> > > >     def __init__(self, strUrl, strContent=None):
> > > >         self.Request    =   urllib2.Request(strUrl, strContent)
> > > >         self.cj.add_cookie_header(self.Request)
> > > >         self.Response   =   urllib2.urlopen(Request)
> > > >         self.cj.extract_cookies(self.Response, self.Request)
> > > >     def url
> > > >     def read(self, intCount):
> > > >         return self.Response.read(intCount)
> > > >     def headers(self, strHeaderName):
> > > >         return self.Response.headers[strHeaderName]
>
> > > > The code does not work because each time SmartRequest is initiated,
> > > > object 'cj' is cleared. How to avoid that?
> > > > The only stupid solution I figured out is use a global CookieJar
> > > > object. Is there anyway that could handle all this INSIDE the class?
>
> > > > I am totally new to OOP & python programming, so could anyone give me
> > > > some suggestions? Thanks in advance
>
> > > Google for urllib2.HTTPCookieProcessor.
>
> > > HTH,
> > > Rob- Hide quoted text -
>
> > > - Show quoted text -
>
> > Wow, thank you Rob Wolfe! Your reply is shortest yet most helpful! I
> > solved this problem by the following code.
>
> > class HTTPRefererProcessor(urllib2.BaseHandler):
> >     """Add Referer header to requests.
>
> >     This only makes sense if you use each RefererProcessor for a
> > single
> >     chain of requests only (so, for example, if you use a single
> >     HTTPRefererProcessor to fetch a series of URLs extracted from a
> > single
> >     page, this will break).
>
> >     There's a proper implementation of this in module mechanize.
>
> >     """
> >     def __init__(self):
> >         self.referer = None
>
> >     def http_request(self, request):
> >         if ((self.referer is not None) and
> >             not request.has_header("Referer")):
> >             request.add_unredirected_header("Referer", self.referer)
> >         return request
>
> >     def http_response(self, request, response):
> >         self.referer = response.geturl()
> >         return response
>
> >     https_request = http_request
> >     https_response = http_response
>
> > def main():
> >     cj = CookieJar()
> >     opener = urllib2.build_opener(
> >         urllib2.HTTPCookieProcessor(cj),
> >         HTTPRefererProcessor(),
> >     )
> >     urllib2.install_opener(opener)
>
> >     urllib2.urlopen(url1)
> >     urllib2.urlopen(url2)
>
> > if "__main__" == __name__:
> >     main()
>
> > And it's working great!
>
> > Once again, thanks everyone!
>
> How does the class HTTPReferrerProcessor do anything useful for you?- Hide 
> quoted text -
>
> - Show quoted text -

Well, it's more browser-like. Many be I should have snipped
HTTPReferrerProcessor code for this discussion.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: n00b with urllib2: How to make it handle cookie automatically?

[est]

On Feb 23, 2:42 am, Rob Wolfe <[EMAIL PROTECTED]> wrote:
> est <[EMAIL PROTECTED]> writes:
> > Hi all,
>
> > I need urllib2 do perform series of HTTP requests with cookie from
> > PREVIOUS request(like our browsers usually do ). Many people suggest I
> > use some library(e.g. pycURL) instead but I guess it's good practise
> > for a python beginner to DIY something rather than use existing tools.
>
> > So my problem is how to expand the urllib2 class
>
> > from cookielib import CookieJar
> > class SmartRequest():
> >     cj=CookieJar()
> >     def __init__(self, strUrl, strContent=None):
> >         self.Request    =   urllib2.Request(strUrl, strContent)
> >         self.cj.add_cookie_header(self.Request)
> >         self.Response   =   urllib2.urlopen(Request)
> >         self.cj.extract_cookies(self.Response, self.Request)
> >     def url
> >     def read(self, intCount):
> >         return self.Response.read(intCount)
> >     def headers(self, strHeaderName):
> >         return self.Response.headers[strHeaderName]
>
> > The code does not work because each time SmartRequest is initiated,
> > object 'cj' is cleared. How to avoid that?
> > The only stupid solution I figured out is use a global CookieJar
> > object. Is there anyway that could handle all this INSIDE the class?
>
> > I am totally new to OOP & python programming, so could anyone give me
> > some suggestions? Thanks in advance
>
> Google for urllib2.HTTPCookieProcessor.
>
> HTH,
> Rob- Hide quoted text -
>
> - Show quoted text -

Wow, thank you Rob Wolfe! Your reply is shortest yet most helpful! I
solved this problem by the following code.

class HTTPRefererProcessor(urllib2.BaseHandler):
"""Add Referer header to requests.

This only makes sense if you use each RefererProcessor for a
single
chain of requests only (so, for example, if you use a single
HTTPRefererProcessor to fetch a series of URLs extracted from a
single
page, this will break).

There's a proper implementation of this in module mechanize.

"""
def __init__(self):
self.referer = None

def http_request(self, request):
if ((self.referer is not None) and
not request.has_header("Referer")):
request.add_unredirected_header("Referer", self.referer)
return request

def http_response(self, request, response):
self.referer = response.geturl()
return response

https_request = http_request
https_response = http_response

def main():
cj = CookieJar()
opener = urllib2.build_opener(
urllib2.HTTPCookieProcessor(cj),
HTTPRefererProcessor(),
)
urllib2.install_opener(opener)

urllib2.urlopen(url1)
urllib2.urlopen(url2)

if "__main__" == __name__:
main()

And it's working great!

Once again, thanks everyone!
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: n00b with urllib2: How to make it handle cookie automatically?

[est]

On Feb 23, 5:57 am, 7stud <[EMAIL PROTECTED]> wrote:
> On Feb 21, 11:50 pm, est <[EMAIL PROTECTED]> wrote:
>
> > Hi all,
>
> > I need urllib2 do perform series of HTTP requests with cookie from
> > PREVIOUS request(like our browsers usually do ).
>
> Cookies from a previous request made in the currently running
> program?  Or cookies from requests that were made when you previously
> ran the program?
>
>
>
>
>
>
>
> > from cookielib import CookieJar
> > class SmartRequest():
> > cj=CookieJar()
> > def __init__(self, strUrl, strContent=None):
> > self.Request=   urllib2.Request(strUrl, strContent)
> > self.cj.add_cookie_header(self.Request)
> > self.Response   =   urllib2.urlopen(Request)
> > self.cj.extract_cookies(self.Response, self.Request)
> > def url
> > def read(self, intCount):
> > return self.Response.read(intCount)
> > def headers(self, strHeaderName):
> > return self.Response.headers[strHeaderName]
>
> > The code does not work because each time SmartRequest is initiated,
> > object 'cj' is cleared. How to avoid that?
> > The only stupid solution I figured out is use a global CookieJar
> > object. Is there anyway that could handle all this INSIDE the class?
>
> Examine this code and its output:
>
> class SmartRequest(object):
> def __init__(self, id):
> if not getattr(SmartRequest, 'cj', None):
> SmartRequest.cj = "I'm a cookie jar. Created by request:

the getattr method is exactly what I am looking for, thanks!


On Feb 23, 2:05 pm, 7stud <[EMAIL PROTECTED]> wrote:
> On Feb 21, 11:50 pm, est <[EMAIL PROTECTED]> wrote:
>
>
>
> > class SmartRequest():
>
> You should always define a class like this:
>
> class SmartRequest(object):
>
> unless you know of a specific reason not to.

Thanks for the advice!
-- 
http://mail.python.org/mailman/listinfo/python-list

n00b with urllib2: How to make it handle cookie automatically?

[est]

Hi all,

I need urllib2 do perform series of HTTP requests with cookie from
PREVIOUS request(like our browsers usually do ). Many people suggest I
use some library(e.g. pycURL) instead but I guess it's good practise
for a python beginner to DIY something rather than use existing tools.

So my problem is how to expand the urllib2 class

from cookielib import CookieJar
class SmartRequest():
cj=CookieJar()
def __init__(self, strUrl, strContent=None):
self.Request=   urllib2.Request(strUrl, strContent)
self.cj.add_cookie_header(self.Request)
self.Response   =   urllib2.urlopen(Request)
self.cj.extract_cookies(self.Response, self.Request)
def url
def read(self, intCount):
return self.Response.read(intCount)
def headers(self, strHeaderName):
return self.Response.headers[strHeaderName]

The code does not work because each time SmartRequest is initiated,
object 'cj' is cleared. How to avoid that?
The only stupid solution I figured out is use a global CookieJar
object. Is there anyway that could handle all this INSIDE the class?

I am totally new to OOP & python programming, so could anyone give me
some suggestions? Thanks in advance
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Could WSGI handle Asynchronous response?

[est]

On Feb 18, 10:35 pm, Jean-Paul Calderone <[EMAIL PROTECTED]> wrote:
> On Mon, 18 Feb 2008 05:27:41 -0800 (PST), est <[EMAIL PROTECTED]> wrote:
> >On Feb 18, 7:05 pm, est <[EMAIL PROTECTED]> wrote:
> >> I am writing a small 'comet'-like app using flup, something like this:
>
> >> def myapp(environ, start_response):
> >>     start_response('200 OK', [('Content-Type', 'text/plain')])
> >>     return ['Flup works!\n']        <-Could this be part
> >> of response output? Could I time.sleep() for a while then write other
> >> outputs?
>
> >> if __name__ == '__main__':
> >>     from flup.server.fcgi import WSGIServer
> >>     WSGIServer(myapp, multiplexed=True, bindAddress=('0.0.0.0',
> >> )).run()
>
> >> So is WSGI really synchronous? How can I handle asynchronous outputs
> >> with flup/WSGI ?
>
> >figured out myself :blush: :blush:
>
> >def demo_app(environ,start_response):
> >    from StringIO import StringIO
> >    stdout = StringIO()
> >    print >>stdout, "Hello world!"
> >    print >>stdout
> >    h = environ.items(); h.sort()
> >    for k,v in h:
> >        print >>stdout, k,'=',`v`
> >    k=start_response("200 OK", [('Content-Type','text/plain')])
> >    for x in range(1, 100):
> >        k(str(x))
> >        time.sleep(1)
> >    return [stdout.getvalue()]
>
> You can do this, but notice that you use up a thread (or a process) for
> each client by doing so.  This means you'll be limited to a fairly small
> number of concurrent clients.
>
> Jean-Paul- Hide quoted text -
>
> - Show quoted text -

So, is there any solution that handles this nicely? Or, more
specificly, is there any solution that is more 'comet'-like, let's say
'socket'-like than WSGI which could almost do full-dulex HTTP
communications, and ready-to-use server & client event-based code
library?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Could WSGI handle Asynchronous response?

[est]

On Feb 18, 7:05 pm, est <[EMAIL PROTECTED]> wrote:
> I am writing a small 'comet'-like app using flup, something like this:
>
> def myapp(environ, start_response):
>     start_response('200 OK', [('Content-Type', 'text/plain')])
>     return ['Flup works!\n']        <-Could this be part
> of response output? Could I time.sleep() for a while then write other
> outputs?
>
> if __name__ == '__main__':
>     from flup.server.fcgi import WSGIServer
>     WSGIServer(myapp, multiplexed=True, bindAddress=('0.0.0.0',
> )).run()
>
> So is WSGI really synchronous? How can I handle asynchronous outputs
> with flup/WSGI ?

figured out myself :blush: :blush:

def demo_app(environ,start_response):
from StringIO import StringIO
stdout = StringIO()
print >>stdout, "Hello world!"
print >>stdout
h = environ.items(); h.sort()
for k,v in h:
print >>stdout, k,'=',`v`
k=start_response("200 OK", [('Content-Type','text/plain')])
for x in range(1, 100):
k(str(x))
time.sleep(1)
return [stdout.getvalue()]

This function-programming style is very odd :-) and
PEP 333 document is so hard for a newbie
-- 
http://mail.python.org/mailman/listinfo/python-list

Could WSGI handle Asynchronous response?

[est]

I am writing a small 'comet'-like app using flup, something like this:


def myapp(environ, start_response):
start_response('200 OK', [('Content-Type', 'text/plain')])
return ['Flup works!\n']<-Could this be part
of response output? Could I time.sleep() for a while then write other
outputs?

if __name__ == '__main__':
from flup.server.fcgi import WSGIServer
WSGIServer(myapp, multiplexed=True, bindAddress=('0.0.0.0',
)).run()


So is WSGI really synchronous? How can I handle asynchronous outputs
with flup/WSGI ?
-- 
http://mail.python.org/mailman/listinfo/python-list

Help! pty interact with bash

[est]

#!/usr/bin/env python
import os, pty, time

class pty_Popen:
def __init__ (self, command, *args):
self.pid, self.fd = pty.fork ()
if self.pid == 0:
os.execv (command, command, args)
else:
pass

def read (self, max_read):
return os.read (self.fd, max_read)

def write (self, text):
return os.write (self.fd, text)

p=pty_Popen("/bin/bash")
p.write("ls --color=always\nexit\n")
print p.read(1024)

I am implementing a wrapper for linux shells with codes above.
This is not responding right, anybody know why?

ps How can I tell which output is stdout or stderr in os.read() ?
-- 
http://mail.python.org/mailman/listinfo/python-list

any python wrapper to call .lib static library(ufmod.lib)?

[est]

I want to play .XM music using Python, now I found ufmod 
http://ufmod.sourceforge.net

It only provide a ufmod.lib to be compiled in C/C++/BASIC, and Python
as a scripting language could not handle these static libraries. What
could I do to with these .lib files?

Cheers
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: dbus-python for windows

[est]

There exists a pre-compiled binary for dbus it was primary wirtten for
deluge win32 port http://www.slurdge.org/deluge-on-windows

There exists a pre-compiled binary for dbus it was primary wirtten for
deluge win32 port http://www.slurdge.org/deluge-on-windows

On Jan 15, 9:14 pm, Suraj Barkale <[EMAIL PROTECTED]> wrote:
> est  gmail.com> writes:
>
> > I am trying to port Scribes to Windows,
>
> Hi there like minded fellow
>
> > sourceforge.net/projects/windbus/≥ but it not for Python, so how could
> > I install dbus module for Windows Python 2.5 ?
>
> I have also started to dabble in windbus-python for the sake of Scribes. I 
> have
> following observations:
> 1. I don't have Visual Studio 2003 (required for compiling Python 2.x modules)
> so alternative is to use MinGW. Follow instructions 
> athttp://js.cx/~justin/mingwYou can skip the GtkGLExt part.
> 2. Checkout windbus from sourceforge, apply the patch (patch.exe doesn't work 
> on
> Vista. Rename it to p4tch.exe)
> 3. Follow the windows build instructions for windbus. It gets compiled 
> correctly
> :).
> 4. dbus-python has dependency on dbus AND dbus-glib. But all my attempts of
> getting dbus-glib to build have failed so for.
> 5. dbus-python looks at wrong place for python headers, this is very easy to
> connect and if we can get dbus-glib to build then we are there :)
>
> I will keep banging my head & let you know once I have cracked it (or the
> dbus-python :)
>
> P.S. Please CC me in your reply as I am not on the list.
>
> Regards,
> Suraj

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: dbus-python for windows

[est]

I am not going to compile anything like that :)  and I only need
Windows library for dbus-python

ps Windows source code is here 
https://windbus.svn.sourceforge.net/svnroot/windbus/trunk/

On Jan 2, 8:44 pm, Jeroen Ruigrok van der Werven <[EMAIL PROTECTED]
nomine.org> wrote:
> -On [20080102 13:41], est ([EMAIL PROTECTED]) wrote:
>
> >I am trying to port Scribes to Windows, but I could not find a package
> >named dbus-python for windows. There is a windbus  >sourceforge.net/projects/windbus/> but it not for Python, so how could
> >I install dbus module for Windows Python 2.5 ?
>
> Well, I assume using the source 
> fromhttp://dbus.freedesktop.org/releases/dbus-python/and trying to build it
> fails on Windows? (I readily admit I have little clue how much of dbus is
> platform-specific.)
>
> --
> Jeroen Ruigrok van der Werven  / asmodai
> イェルーン ラウフロック ヴァン デル ウェルヴェンhttp://www.in-nomine.org/|http://www.rangaku.org/
> Resolve to find thyself; and to know that he who finds himself, loses
> his misery...

-- 
http://mail.python.org/mailman/listinfo/python-list

dbus-python for windows

[est]

Hi all

I am trying to port Scribes to Windows, but I could not find a package
named dbus-python for windows. There is a windbus  but it not for Python, so how could
I install dbus module for Windows Python 2.5 ?

ps Is there anyone trying to port Scribes to Windows?
-- 
http://mail.python.org/mailman/listinfo/python-list

Any library parsing wikipedia API

[est]

I got some text via wikipedia API

http://en.wikipedia.org/w/api.php?action=query&prop=revisions&titles=Chengdu&rvprop=content&redirects&format=xml&callback=wikiCallback

but it's not HTML formated. So how can I parse these texts to standard
HTML? Are there any library in python could do this?

-- 
http://mail.python.org/mailman/listinfo/python-list

Need suggestion on a GTalk Webservice

[est]

I am writing a GTalkBot which provides a webservice
http://xxx.com/services/sendmessage.html

If someone POST some text to http://xxx.com/services/sendmessage.html
it will transfer the text to some friends on GTalk.

I tried using django's __init__.py function to build this service but
it seemed too stupid, here is the discuss thread:
http://groups.google.com/group/django-users/t/9e652d3ad37338fc

I thought about other solutions, running a GTalk daemon process and
using sockets to interact with django, but it seemed to COMPLEX to me.
Is there a simpler way to do this? a pythonic, simple way?

-- 
http://mail.python.org/mailman/listinfo/python-list

set_proxy in urllib2 is driving my crazy

[est]

A simple py script

import urllib2
req=urllib2.Request("http://www.google.com";)
req.set_proxy("127.0.0.1:1","http")
print urllib2.urlopen(req).read()




Here is error:

 "C:\Python25\python.exe" -u "E:\Python\Crawler\test.py"


Traceback (most recent call last):
  File "E:\Python\Crawler\test.py", line 4, in 
print urllib2.urlopen(req).read()
  File "C:\Python25\lib\urllib2.py", line 121, in urlopen
return _opener.open(url, data)
  File "C:\Python25\lib\urllib2.py", line 374, in open
response = self._open(req, data)
  File "C:\Python25\lib\urllib2.py", line 392, in _open
'_open', req)
  File "C:\Python25\lib\urllib2.py", line 353, in _call_chain
result = func(*args)
  File "C:\Python25\lib\urllib2.py", line 1101, in http_open
return self.do_open(httplib.HTTPConnection, req)
  File "C:\Python25\lib\urllib2.py", line 1076, in do_open
raise URLError(err)
urllib2.URLError: 


I built a proxy with CCproxy 6.3.2 on my own machine, and it's working
absolutely all right, but how can my simple python script go wrong ?

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Newbie prob: How to write a file with 3 threads?

[est]

On May 8, 2:13 pm, Dennis Lee Bieber <[EMAIL PROTECTED]> wrote:
> On 7 May 2007 18:22:07 -0700, est <[EMAIL PROTECTED]> declaimed
> the following in comp.lang.python:
>
>
>
> > I guess I will write multiple files, one thread one file, and after
> > all threads terminates, I combine the files. That's a cheaper
> > solution, I think.
>
> Given your first description:
>
> > I need to write a file using 3 threads simutaniously, e.g. Thread 1
> > write the first byte of test.bin with an "a", second thread write the
> > second byte "b", third thread write the third byte "c". Anyone could
> > give a little example on how to do that?
>
> ... any other solution may not have worked anyway. That is, if you
> really expect the three threads to interleave their output as
> abcabcabcabc. If threading, you have no assurance that any single thread
> will follow any other during a task switch. It all depends upon where a
> task switch takes place.
>
> But then, your example isn't too clear of what you really are
> producing for output. If, instead of "bytes", you meant that each thread
> was writing logging information, the solution would be to use the
> logging module -- so far as I know, the logging module /does/ perform
> the needed access locking.
>
> OTOH, if you really mean to have three separate byte producers,
> interleaving output, the only safe way would be to have /each/ have an
> output Queue, and a fourth thread doing the writing using a loop of the
> form:
>
> while True:
> a = aQueue.get()
> fout.write(a)
> b = bQueue.get()
> fout.write(b)
> c = cQueue.get()
> fout.write(c)
>
> Using the separate queues means that the writer WILL wait until the
> next producer in the interleave has produced its data. Of course, this
> structure has the drawback that all producers must produce the same
> amount of data -- otherwise it blocks forever on the queue from the
> producer has stopped generating data.
> --
> WulfraedDennis Lee Bieber   KD6MOG
> [EMAIL PROTECTED] [EMAIL PROTECTED]
> HTTP://wlfraed.home.netcom.com/
> (Bestiaria Support Staff:   [EMAIL PROTECTED])
> HTTP://www.bestiaria.com/

I'd like to say VERY VERY VERY thank you for your detailed
information, that's a lot encourage for a beginner. In fact I am
writing a multi thread download ultility, and I found that very hard
for me. Can you recommand any sample code where I can start with? I
hope I can catch up with everyone here, I'll try my best learning
python. Thank you again.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Newbie prob: How to write a file with 3 threads?

[est]

On May 8, 1:29 am, Dennis Lee Bieber <[EMAIL PROTECTED]> wrote:
> On 7 May 2007 00:13:46 -0700, est <[EMAIL PROTECTED]> declaimed
> the following in comp.lang.python:
>
>
>
> > I'll try Queue.Queue, thank you. I didn't expect that multithread
> > write a file is so troublesome
>
> Multiple writers to a file, threaded or not (ie, multiple processes,
> hosts on network, etc.), is always a pain unless one can ensure that all
> writers use a common locking mode to prevent overlapping calls. If all
> the writers are internal to one program, one can implement internal
> locks -- if external one needs OS support for multi-process locking (VMS
> common event flag clusters, for example). Easier to dedicate one
> (internal) writer and use the naturally locked Queue to control access.
> --
> WulfraedDennis Lee Bieber   KD6MOG
> [EMAIL PROTECTED] [EMAIL PROTECTED]
> HTTP://wlfraed.home.netcom.com/
> (Bestiaria Support Staff:   [EMAIL PROTECTED])
> HTTP://www.bestiaria.com/

I guess I will write multiple files, one thread one file, and after
all threads terminates, I combine the files. That's a cheaper
solution, I think.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Newbie prob: How to write a file with 3 threads?

[est]

On May 7, 5:12 am, MRAB <[EMAIL PROTECTED]> wrote:
> On May 6, 9:51 am, Marc 'BlackJack' Rintsch <[EMAIL PROTECTED]> wrote:> In 
> <[EMAIL PROTECTED]>, est wrote:
> > > I need to write a file using 3 threads simutaniously, e.g. Thread 1
> > > write the first byte of test.bin with an "a", second thread write the
> > > second byte "b", third thread write the third byte "c". Anyone could
> > > give a little example on how to do that?
>
> > Simplest solution is: don't do that.  Write from one thread and send the
> > date from the other threads via a `Queue.Queue` to the writing thread.
> > Send the number of the thread with the data so the writer thread knows in
> > which order the data has to be written.
>
> [snip]
> Or have a `Queue.Queue` for each source thread and make the writer
> thread read from each in turn.


I'll try Queue.Queue, thank you. I didn't expect that multithread
write a file is so troublesome

-- 
http://mail.python.org/mailman/listinfo/python-list

Newbie prob: How to write a file with 3 threads?

[est]

I need to write a file using 3 threads simutaniously, e.g. Thread 1
write the first byte of test.bin with an "a", second thread write the
second byte "b", third thread write the third byte "c". Anyone could
give a little example on how to do that?

I have my code, but it makes python intepreter crash everytime on my
Vista.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: py script modify it's Modified Time?

[est]

On 4月12日, 上午2时26分, "Gabriel Genellina" <[EMAIL PROTECTED]> wrote:
> En Wed, 11 Apr 2007 10:33:57 -0300, <[EMAIL PROTECTED]> escribió:
>
> > On Apr 11, 6:14 am, "est" <[EMAIL PROTECTED]> wrote:
> >> I guess os.stat() could retrieve information about ctime, mtime of a
> >> path, but how can I change them? Is it possible to make it work both
> >> under Win32 and Linux?
>
> > I am not sure if you can change a creation time of a file on NT, but
> > the rest sounds possible; according to this post anyway:
>
> >http://mail.python.org/pipermail/python-list/2001-August/100214.html
>
> That post is rather old... With os.utime you can set the access and  
> modified time, and it's portable.
> Creation time isn't stored on [the normal filesystems used by] Linux, and  
> on Windows you can use win32file.SetFileTime
>
> > It mentions using the win32 modules, which can be found here:
> >http://aspn.activestate.com/ASPN/docs/ActivePython/2.4/pywin32/win32_...
>
> Betterhttps://sourceforge.net/projects/pywin32/
>
> --
> Gabriel Genellina

Thanks guys ,you are of great healp!

-- 
http://mail.python.org/mailman/listinfo/python-list

py script modify it's Modified Time?

[est]

I guess os.stat() could retrieve information about ctime, mtime of a
path, but how can I change them? Is it possible to make it work both
under Win32 and Linux?

P.S. How to get the current running python?  (Like
WScript.ScriptFullName in vbscript)

-- 
http://mail.python.org/mailman/listinfo/python-list