Re: Sqlite pragma statement "locking_mode" set to "EXCLUSIVE" by default

[Ryan Stuart]

On Tue, Sep 22, 2015 at 5:12 PM, Sol T  wrote:

> I know I can do this per connection, however how can I have it set to
> default? Does this need to be compiled into python?
>

If you need to compile sqlite you might want to look at apsw:
https://github.com/rogerbinns/apsw

Cheers


>
> On Tue, Sep 22, 2015 at 2:04 PM, Ryan Stuart 
> wrote:
>
>> On Thu, Sep 17, 2015 at 2:24 PM, sol433tt  wrote:
>>
>>> I would like to have the Sqlite pragma statement "locking_mode" set to
>>> "EXCLUSIVE" by default (RO database). Does this need to be compiled in? How
>>> might this be achieved?
>>>
>>
>> You can issue any PRAGA statement you like using the execute method on a
>> connection as per the documentation (
>> https://docs.python.org/2/library/sqlite3.html#sqlite3.Connection.execute).
>> For information on the specific PRAGMA option you need, see
>> https://www.sqlite.org/pragma.html#pragma_locking_mode. I can't see any
>> mention of it being a compile time PRAGMA.
>>
>> Cheers
>>
>>
>>>
>>> There is some performance to be gained. I have a number of python
>>> scripts, and don't want to alter the pragma statement for every script.
>>> This computer never writes to the databases.
>>>
>>> thank you
>>>
>>> --
>>> https://mail.python.org/mailman/listinfo/python-list
>>>
>>>
>>
>>
>> --
>> Ryan Stuart, B.Eng
>> Software Engineer
>>
>> W: http://www.kapiche.com/
>> M: +61-431-299-036
>>
>
>


-- 
Ryan Stuart, B.Eng
Software Engineer

W: http://www.kapiche.com/
M: +61-431-299-036
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Sqlite pragma statement "locking_mode" set to "EXCLUSIVE" by default

[Ryan Stuart]

On Thu, Sep 17, 2015 at 2:24 PM, sol433tt  wrote:

> I would like to have the Sqlite pragma statement "locking_mode" set to
> "EXCLUSIVE" by default (RO database). Does this need to be compiled in? How
> might this be achieved?
>

You can issue any PRAGA statement you like using the execute method on a
connection as per the documentation (
https://docs.python.org/2/library/sqlite3.html#sqlite3.Connection.execute).
For information on the specific PRAGMA option you need, see
https://www.sqlite.org/pragma.html#pragma_locking_mode. I can't see any
mention of it being a compile time PRAGMA.

Cheers


>
> There is some performance to be gained. I have a number of python scripts,
> and don't want to alter the pragma statement for every script. This
> computer never writes to the databases.
>
> thank you
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
>


-- 
Ryan Stuart, B.Eng
Software Engineer

W: http://www.kapiche.com/
M: +61-431-299-036
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Parser needed.

[Ryan Stuart]

On Tue, Jun 9, 2015 at 12:30 PM, Skybuck Flying 
wrote:

> Anyway... I am trying a more robust parser... because my own parser right
> now didn't work out for new inputs.
>

You should take a look at lrparsing:
https://pypi.python.org/pypi/lrparsing/1.0.11

Cheers


>
> It almost worked except for first item... don't know what problem was
> maybe just this...
>
> But I'll try and do it the usually way.
>
> "Tokenize", "Parse" etc.
>
> It's significantly slower though.
>
> Maybe an idea for a new kind of parser could be:
>
> Find all recgonized token keywords in one go... and stuff all find indexes
> into a list... and then perhaps sort the list...
>
> and then use the sorted list... to return token found at that position...
>
> and then start processing like that...
>
> Benefit of this idea is less characters to compare... and could be done
> with just integer compare... or even lookup table processing.
>
> Added benefit is still in-order which is nice.
>
> For now I won't do it... cause I am not sure it would be an improvemt.
>
> Another idea which might be an improvement is.
>
> Parallel searching... ofcourse that might not be possible... though...
> multi core does it exist and threading too.
>
> But to mimic parallel searching and to prevent problems.
>
> A sliding window approach could be taken.
>
> And perhaps items found that way in a certain buffer or so and then still
> added to processing list or so...
>
> which kinda mimics parallel search... but just uses data cache nicely.
>
> Though it's somewhat of a complex method... I will avoid it for now.
>
> The split() routine is real nice... to get rid of fudd/white space... and
> just tokens which is nice.
>
> So for now I will use that as my tokenizer ;) =D
>
> and bracket level counting and sections and stuff like that yeah...
>
>
> Bye,
>  Skybuck.
> --
> https://mail.python.org/mailman/listinfo/python-list
>



-- 
Ryan Stuart, B.Eng
Software Engineer

ABN: 81-206-082-133
W: http://www.textisbeautiful.net/
M: +61-431-299-036
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: test2

[Ryan Stuart]

et_tally
> from django.http import HttpResponsePermanentRedirect
> from django.shortcuts import render_to_response
> from django.contrib import admin
> admin.autodiscover()
>
> import os
>
> PROJECT_PATH = os.path.realpath(os.path.dirname(__file__))
>
> # Uncomment the next two lines to enable the admin:
> from django.contrib import admin
> admin.autodiscover()
>
> urlpatterns = patterns('asset.views',
> # Examples:
> # url(r'^$', 'ipdb.views.home', name='home'),
> # url(r'^ipdb/', include('ipdb.foo.urls')),
>
> # Uncomment the admin/doc line below to enable admin documentation:
>
> # Uncomment the next line to enable the admin:
> url(r'^admin/', include(admin.site.urls)),
> #(r'^accounts/', include('registration.urls')),
> (r'^$', lambda request:
> HttpResponsePermanentRedirect('/asset/overview')),
> (r'^heatmap/', country_tally),
> (r'^grid/',grid),
> (r'^nto/',newest_to_oldest),
> (r'^ctwtt/',country_tally_web_top10),
> (r'^ctw/',country_tally_web),
> (r'^at/',asset_tally),
>
>  #(r'^/$', ipdb_overview), #Start
>  #(r'^$', ipdb_overview), #Start
> (r'^asset/overview/$', ipdb_overview), #Start
> (r'^input/$', gui_add),
> (r'^search/$', gui_search),
> (r'^search$', gui_search),
> # (r'^delete/$', ipdb_input_delete),
> (r'^api/add/$', api_add), #check sensornet urls.py and
> views/sensors.py for cool tricks.
> (r'^api/search/$', api_search), #check sensornet urls.py and
> views/sensors.py for cool tricks.
> (r'^api/file/$', api_file),
>
> #(r'^asset/media/(?P.*)$', 'django.views.static.serve',
> {'document_root':  os.path.join(PROJECT_PATH, '../asset/media')}),
>
> #(r'^ip_db/media/(?P.*)$', 'django.views.static.serve',
> {'document_root':  os.path.join(PROJECT_PATH, 'ip_db/media')}),
> #(r'^media/(?P.*)$', 'django.views.static.serve',
> {'document_root':  os.path.join(PROJECT_PATH, 'media')}),
> #(r'^static/(?P.*)$', 'django.views.static.serve',
> {'document_root':  '/home/sensornet/static'}),
>
> # extjs4 stuff
> #(r'^asset/overview/extjs/(?P.*)$', 'serve', {'document_root':
> os.path.join(PROJECT_PATH, '../asset/extjs')}),
> #(r'^extjs/(?P.*)$', 'django.views.static.serve',
> {'document_root':  os.path.join(PROJECT_PATH, '../ip_db/extjs')}),
> #(r'^extjs/(?P.*)$', 'django.views.static.serve',
> {'document_root':  '/home/totis/tree/ipdb/django/ipdb/asset/static/extjs'}),
> ) + static(settings.MEDIA_URL, document_root=settings.MEDIA_ROOT),
> 'ipdb.asset': {
> 'handlers': ['console', 'logfile'],
> 'level': 'DEBUG',
> },
> }
> }
>
> ===
>
>
>
> --
> https://mail.python.org/mailman/listinfo/python-list
>



-- 
Ryan Stuart, B.Eng
Software Engineer

ABN: 81-206-082-133
W: http://www.textisbeautiful.net/
M: +61-431-299-036
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: HELP!! How to ask a girl out with a simple witty Python code??

[Ryan Stuart]

On Thu, 5 Mar 2015 at 11:35 Xrrific  wrote:

> Could you please come up with something witty incorporating a simple
> python line like If...then... but..etc.
>

Send her this:

import base64
print(base64.b64decode('CkhpLCAKVGhpcyBpcyBzb21lb25lIGZyb20gdGhlIHB5dGhvbiBtYWlsaW5nIGxpc3QuIFRoaXMgZ3V5IChoZSBwb3N0ZWQgdW5kZXIgdGhlIG5hbWUgYFhycmlmaWNgKSBwb3N0ZWQgaGVyZSBhc2tpbmcgZm9yIHNvbWUgY29kZSB0aGF0IHdvdWxkIGltcHJlc3MgeW91LiBJdCB3YXMgdmVyeSB0ZW1wdGluZyB0byBwdXQgc29tZXRoaW5nIHZlcnkgb2ZmZW5zaXZlIGluIHRoaXMgbWVzc2FnZSEgUmF0aGVyLCBoZXJlIGlzIHNvbWUgYXNjaWkgYXJ0LgoKICAgICAgICAgICAsCiAgICAgICwsICAgJSUgICAsICAgLAogICAgIDolJSU7IDslJSUgOyUgICUlOgogICAgICA6JSUlICUlJSUgJSUgJSU6ICAgCiAgICAgOiUlJSUlICUlJSUgJSAlJSU6IAogICAgIDolJSUlJSUgJSUlJSAlJSUlOgogICAgICA6JSUlJSUgJSUlICUlJSU6CiAgICAgICA6JSUlJSAlJSAlJSUlOgogICAgICAgIDolJSUlJSAlJSUlOgogICAgICAgICA6JSUlJSAlJSU6IAogICAgICAgICAgJiU6JSY6JSYKICAgICAgICAgICAmJjomOiYKICAgICAgICAgICAgJiYmJgogICAgICAgICAgICA7JiY7ICAgICA7CiAgICAgICAgICAgICA6OiAgICA7Ozs7CiAgICAgICAgICAgICA6OiAgIDs7Ozs7CiAgICAgICAgICAgICA6OiAgOzs7Ozs7CiAgICAgICAgICAgICA6OiA7Ozs7OzsKICAgICAgIDsgICAgIDo6IDs7Ozs7CiAgICAgIDs7ICAgICA6Ojs7OzsgCiAgICAgOzs7OyAgICA6OjsgCiAgICAgOzs7OzsgICA6OgogICAgIDs7Ozs7OyAgOjoKICAgICAgOzs7OzsgIDo6CiAgICAgICAgOzs7OyA6OgogICAgICAgICAgIDs7OjoKICAgICAgICAgICAgOzo6CiAgICAgICAgICAgICA6OgogICAgICAgICAgICAgOjoKICAgICAgICAgICAgIDo6CiAgICAgICAgICAgICA6OgogICAgICAgICAgICAgOjoKICAgICAgICAgICAgIDo6CiAgICAgICAgICAgICA6OgogICAgICAgICAgICAgOjoKClAuUy4gWWVzIGl0IGlzIGEgbGl0dGxlIGxhbWUgdGhhdCBoZSBwb3N0ZWQgb24gdGhlIG1haWxpbmcgbGlzdCBmb3IgdGhpcywgYnV0IGl0J3MgdGhlIHRob3VnaHQgdGhhdCBjb3VudHMhCg=='))

Cheers


>
> You will make me a very happy man!!!
>
> Thank you very much!!!
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Are threads bad? - was: Future of Pypy?

[Ryan Stuart]

On Tue Feb 24 2015 at 3:32:47 PM Paul Rubin  wrote:

> Ryan Stuart  writes:
> Sure, the shared memory introduces the possibility of some bad errors,
> I'm just saying that I've found that by staying with a certain
> straightforward style, it doesn't seem difficult in practice to avoid
> those errors.
>

And all I'm saying is that it doesn't matter how careful you are, all it
takes is someone up the stack to not be as careful and you will be bitten.
And even if they are careful, it's still very easy to be bitten because
everything runs in shared memory.


> I don't understand what you mean about malloc.
>

My point is malloc, something further up (down?) the stack, is making
modifications to shared state when threads are involved. Modifying shared
state makes it infinitely more difficult to reason about the correctness of
your software.


> That article was interesting in some ways but confused in others.  One
> way it was interesting is it said various non-thread approaches (such as
> coroutines) had about the same problems as threads.  Some ways it
> was confused were:
>

We clearly got completely different things from the article. My
interpretation was that it was making *the exact opposite* point to what
you stated mainly because non-threading approaches don't share state. It
states that quite clearly. For example "it is – literally – exponentially
more difficult to reason about a routine that may be executed from an
arbitrary number of threads concurrently".


>   1) thinking Haskell threads were like processes with separate address
>   spaces.  In fact they are in the same address space and programming
>   with them isn't all that different from Python threads, though the
>   synchronization primitives are a bit different.  There is also an STM
>   library available that is ingenious though apparently somewhat slow.
>

I don't know Haskell, so I can't comment too much, except to say that by
default Haskell looks to use lightweight threads where only 1 thread can be
executing at a time [1] (sounds similar to the GIL to me). That doesn't
seem to be shared state multithreading, which is what the article is
referring to. But again, they will undoubtedly be someone sharing state
higher up the stack.


>   2) it has a weird story about the brass cockroach, that basically
>   signified that they didn't have a robust enough testing system to be
>   able to reproduce the bug.  That is what they should have worked on.
>

The point was that it wasn't feasible to have a robust testing suite
because, you guessed it, shared state and concurrent threads producing n*n
complexity.


>   3) It goes into various hazards of the balance transfer example not
>   mentioning that STM (available in Haskell and Clojure) completely
>   solves it.
>

This is probably correct. Is there any STM implementations out that that
don't significantly compromise performance?

Anyway, I got one thing out of this, which is that the multiprocessing
> module looks pretty nice and I should try it even when I don't need
> multicore parallelism, so thanks for that.
>

It's 1 real advantage is that it side-steps the GIL. So, if you need to
utilise multiple cores for CPU bound tasks, then it might well be the only
option.

Cheers


> In reality though, Python is still my most productive language for
> throwaway, non-concurrent scripting, but for more complicated concurrent
> programs, alternatives like Haskell, Erlang, and Go all have significant
> attractions.
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Future of Pypy?

[Ryan Stuart]

On Mon Feb 23 2015 at 4:15:42 PM Paul Rubin  wrote:
>
> What do you mean about Queues working with processes?  I meant
> Queue.Queue.  There is multiprocessing.Queue but that's much less
> capable, and it uses cumbersome IPC like pipes or sockets instead of a
> lighter weight lock.  Threads can also share read-only data and you can
> pass arbitrary objects (such as code callables that you want the other
> thread to execute--this is quite useful) through Queue.Queue.  I don't
> think you can do that with the multiprocessing module.
>

These things might be convenient but they are error prone for the reasons
pointed out. Also, the majority can be achieved via the process approach.
For example, using fork to take a copy of the current process (including
the heap) you want to use will give you access to any callables on the
heap.

The vital point here is that fork takes a *copy* of the process and runs in
a *separate* memory space. This means there can be no accidents here. If it
were to run in the same memory space like a thread then bugs anywhere in
the code you run could cause very nasty problems. This includes not only
bugs in your code, but also bugs in any other library code. Not only are
they nasty, they could be close to invisible.

And when I saw any code you run, I literally mean any. Even if you are
extra careful to not touch any shared state in your code, you can almost be
guaranteed that code higher up the stack, like malloc for example, *will*
be using shared state.

The risk of unintended and difficult to track issues when using threads is
very high because of shared state. Even if you aren't sharing state in your
code directly, code higher up the stack will be sharing state. That is the
whole point of a thread, that's what they were invented for. Using threads
safely might well be impossible much less verifiable. So when there are
other options that are just as viable/functional, result in far less risk
and are often much quicker to implement correctly, why wouldn't you use
them? If it were easy to use threads in a verifiably safe manner, then
there probably wouldn't be a GIL.

Cheers


> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Future of Pypy?

[Ryan Stuart]

On Mon Feb 23 2015 at 1:50:40 PM Paul Rubin  wrote:

> That article is about the hazards of mutable state shared between
> threads.  The key to using threads safely is to not do that.  So the
> "transfer" example in the article would instead be a message handler in
> the thread holding the account data, and it would do the transfer in the
> usual sequential way.  You'd start a transfer by sending a message
> through a Queue, and get back a reply through another queue.
>

I think that is a pretty accurate summary. In fact, the article even says
that. So, just to iterate its point, if you are using non-blocking Queues
to communicate to these threads, then you just have a communicating event
loop. Given that Queues work perfectly with with processes as well, what is
the point of using a thread? Using a process/fork is far safer in that
someone can't "accidentally" decide to alter mutable state in the future.


> You might like this:
>
> http://jlouisramblings.blogspot.com/2012/08/getting-
> 25-megalines-of-code-to-behave.html


Thanks for this, I'll take a look.

Cheers


>
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Future of Pypy?

[Ryan Stuart]

On Mon Feb 23 2015 at 12:05:46 PM Paul Rubin 
wrote:

> I don't see what the big deal is.  I hear tons of horror stories about
> threads and I believe them, but the thing is, they almost always revolve
> around acquiring and releasing locks in the wrong order, forgetting to
> lock things, stuff like that.  So I've generally respected the terror
> and avoided programming in that style, staying with a message passing
> style that may take an efficiency hit but seems to avoid almost all
> those problems.  TM also helps with lock hazards and it's a beautiful
> idea--I just haven't had to use it yet.  The Python IRC channel seems to
> rage against threads and promote Twisted for concurrency, but Twisted
> has always reminded me of Java.  I use threads in Python all the time
> and haven't gotten bitten yet.
>
>
Many people have written at length about why it's bad. The most recent
example I have come across is here ->
https://glyph.twistedmatrix.com/2014/02/unyielding.html

It's not a specific Python problem. I must be in the limited crowd that
believes that the GIL is a *feature* of Python. Then again, maybe it isn't
so limited since the GIL-less python implementations haven't really taken
off.

I have yet to come across a scenario I couldn't solve with either
Processes, NumPy or event loops. Yes, when using processes, the passing of
data can be annoying sometimes. But that is far less annoying then trying
to debug something that shares state across threads.

It's great that you haven't been bitten yet. But, the evidence seems to
suggest the either you *will* be bitten at some point or, you already have
been, and you just don't know it.

Cheers


> > Writing multithreaded code is *hard*. It is not a programming model
> > which comes naturally to most human beings. Very few programs are
> > inherently parallelizable, although many programs have *parts* which
> > can be successfully parallelized.
>
> Parallel algorithms are complicated and specialized but tons of problems
> amount to "do the same thing with N different pieces of data", so-called
> embarassingly parallel.  The model is you have a bunch of worker threads
> reading off a queue and processing the items concurrently.  Sometimes
> separate processes works equally well, other times it's good to have
> some shared data in memory instead of communicating through sockets.  If
> the data is mutable then have one thread own it and access it only with
> message passing, Erlang style.  If it's immutable after initialization
> (protect it with a condition variable til initialization finishes) then
> you can have read-only access from anywhere.
>
> > if you're running single-thread code on a single-core machine and
> > still complaining about the GIL, you have no clue.
>
> Even the Raspberry Pi has 4 cores now, and fancy smartphones have had
> them for years.  Single core cpu's are specialized and/or historical.
>
> > for some programs, the simplest way to speed it up is to vectorize the
> > core parts of your code by using numpy.  No threads needed.
>
> Nice for numerical codes, not so hot for anything else.
>
> > Where are the people flocking to use Jython and IronPython?
>
> Shrug, who knows, those implementations were pretty deficient from what
> heard.
>
> > For removal of the GIL to really make a difference: ...
> > - you must be performing a task which is parallelizable and not
> inherently
> > sequential (no point using multiple threads if each thread spends all its
> > time waiting for the previous thread);
>
> That's most things involving concurrency these days.
>
> > - the task must be one that moving to some other multi-processing model
> > (such as greenlets, multiprocess, etc.) is infeasible;
>
> I don't understand this--there can be multiple ways to solve a problem.
>
> > - your threading bottleneck must be primarily CPU-bound, not I/O bound
> > (CPython's threads are already very effective at parallelising I/O
> tasks);
>
> If your concurrent program's workload makes it cpu-bound even 1% of the
> time, then you gain something by having it use your extra cores at those
> moments, instead of having those cores always do nothing.
>
> > - and you must be using libraries and tools which prevent you moving to
> > Jython or IronPython or some other alternative.
>
> I don't get this at all.  Why should I not want Python to have the same
> capabilities?
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: TypeError: list indices must be integers, not tuple

[Ryan Stuart]

Hi,

There is a lot of issues with this code. First, setting fav to a 1 tuples
with a string probably isn't what you want. What you probably mean is:

if restraunt == ("Pizza"):
fav = 1

Second, when you are trying to lookup items in Menu, you are using the
incorrect fav. Lists have int indicies (just like the error points out).
Values like ("1") aren't integers.

Thirdly, Menu is a list of lists. To fetch "Barbeque pizza" from Menu, you
need to do Menu[0][0], not Menu[0, 0].

Finally, Python comes with a style guide which you can find in pep8
. Your code violates that guide
in many places. It might be worth working through the Python Tutorial
.

Cheers

On Tue Feb 10 2015 at 9:55:40 AM  wrote:

> import random
> RandomNum = random.randint(0,7)
> restraunt = raw_input("What's your favourite takeaway?Pizza, Chinease or
> Indian?")
> if restraunt == ("Pizza"):
> fav = ("1")
>
> elif restraunt == ("Chinease"):
> fav = ("2")
>
> elif restraunt == ("Indian"):
> fav = ("3")
>
> else:
> print("Try using a capital letter, eg; 'Chinease'")
>
> Menu = [["Barbeque pizza","Peparoni","Hawain"],["
> Curry","Noodles","Rice"],["Tika Masala","Special Rice","Onion Bargees"]]
>
> print Menu[fav,RandomNum]
>^
> TypeError: list indices must be integers, not tuple
>
> How do I set a variable to a random number then use it as a list indece,
> (I'm only a student in his first 6 months of using python)
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Profiler for long-running application

[Ryan Stuart]

Hi Asad,

Is there any reason why you can't just use profile/cProfile? In particular,
you could use the api of that module to save out the profile stats to an
external file with a unique name and then inspect them later with a tool
like snakeviz . The code to save
profile stats might look like the following:

pr = cProfile.Profile()
pr.runcall(your_celery_task_without_async)
ps = pstats.Stats(pr)
ps.dump_stats("my_task.profile")

Obviously you need to call your celery task function directly, not via
Celery using delay() or any derivative. Alternatively, you could try
something like line_profiler  by
again, calling the task directly.

If using this method it turn out your actual task code is running quite
fast, then I'd suggest that the majority of the time is being lost in
transferring the task to the Celery node.

Cheers

On Mon Feb 09 2015 at 5:20:43 AM Asad Dhamani  wrote:

> I have a Flask application where I run a specific task asynchronously
> using Celery. Its basically parsing some HTML and inserting data into a
> Postgres database(using SQLAlchemy). However, the task seems to be running
> very slowly, at 1 insert per second.
> I'd like to find out where the bottleneck is, and I've been looking for a
> good profiler that'd let me do this, however, I couldn't find anything. Any
> recommendations would be great.
> --
> https://mail.python.org/mailman/listinfo/python-list
>
-- 
https://mail.python.org/mailman/listinfo/python-list

Convert 3d NumPy array into 2d

[phinn stuart]

Hi everyone, how can I convert (1L, 480L, 1440L) shaped numpy array into
(480L, 1440L)?

Thanks in the advance.

phinn
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: I look for a list to convert time zone abbreviation to full time zone in python

[Stuart Bishop]

On Mon, Dec 2, 2013 at 11:18 PM, Stéphane Klein
 wrote:

> * are there the same list somewhere (I didn't found in pytz) ?

Not that I know of.

> * is it possible to append this list in pytz or in standard python date 
> module ?

It could go into pytz (but generated from the IANA database, not from
the list you quote). Whether it should go into pytz is debatable.

If you need to map an abbreviation back to a single timezone you are
solving the wrong problem, because you can only map an abbreviation
back to a list of possible timezones (And that list might change when
the database is corrected). Also, to correctly represent this you need
to specify the point in time. EST in 'Tue Dec  3 20:44:00 EST 2013'
maps to about 3 timezones. EST in 'Tue Dec  3 20:44:00 EST 2011' maps
to about 6.

-- 
Stuart Bishop 
http://www.stuartbishop.net/
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: OAuth 2.0 implementation

[Stuart Bishop]

On Tue, Mar 27, 2012 at 10:11 AM, Ben Finney  wrote:
> Demian Brecht  writes:
>
>> I'm getting close to an alpha release of an OAuth 2.0 implementation
>> (https://github.com/demianbrecht/py-sanction).
>
> Thank you for doing this work.
>
> As someone who uses OpenID, what can I read about why OAuth is better?

They are different, and often you need to use both.

OpenID allows web sites to authenticate someone. It is not really
useful for anything not an interactive web site. The consuming site
never gets your keys, it just gets confirmation from the provider that
the user is who they claim they are and maybe some details that the
provider chooses to provide such as an email address.

OAuth is for generating authentication keys that allow a program to
authenticate as someone and perform operations on their behalf. You
use OAuth to generate a key so that Foursquare can send messages via
Twitter on your behalf, or so the Facebook client on your phone can
access your account without storing your password. You also get
authentication here, as you can't generate a key without being
authenticated, but the real reason it is used instead of OpenID is so
you can keep the key and keep using it to act as the user; you can
keep using that key until it expires or it is revoked.

Authentication providers that don't provide a webapi just implement
OpenID. Big sites like Google and Facebook implement both OpenID (for
'log in with your GMail account') and OAuth ('post this message to
your Facebook wall').

-- 
Stuart Bishop 
http://www.stuartbishop.net/
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Web Applications - Separating DB Connections

[Stuart MacKay]

A simple way to do this is use fabric for deployment. It allows you to 
upload a file as if it was a template and replaces any placeholder 
strings with values supplied when you upload. The values can be supplied 
either in a config file or interactively when the deployment takes place.


For my django deployments to a production server I have the database 
connection information in a config file that is separate from the app 
source. The name of the config file is passed on the command line when 
running the deployment.


See fabric.contrib.files.upload_template on 
http://docs.fabfile.org/en/1.0.1/api/contrib/files.html

and the --config option on http://docs.fabfile.org/en/1.0.1/usage/fab.html

Stuart MacKay
Lisbon, Portugal


Hello Everyone:

   I am trying to find a way to extract and remove database connection
information (username, password, schema name) from the application
source. I need to do this because in my organization - for security
reasons - access to databases is controlled by a separate department;
and as such, when a solution is deployed to production - the
authentication credentials for the databases are changed (and not told
to the development team).

   Currently all development is done in Java and with that they have
the ability to publish databases as a service in their application
server; this way users can be granted access to modify the credentials
to the JDBC data source without having to edit source code of the
application being deployed.  I am looking for something similar in
Python (short of using Jython).

Thanks!


--
http://mail.python.org/mailman/listinfo/python-list

Re: FW: help please

[Stuart MacKay]

On 18/05/2011 12:47, Albert Hopkins wrote:

On Tue, 2011-05-17 at 21:46 -0300, Gabriel Genellina wrote:

En Tue, 17 May 2011 16:48:29 -0300, Albert Hopkins
  escribió:

On Tue, 2011-05-17 at 10:18 -0600, Littlefield, Tyler wrote

Not to be pedantic or anything, and I may not be able to help
regardless, but it looks like your space key is fixed, and I don't
really care to pick through and try to play hangman with your message.

I actually, at first glance, thought it was spam, ignored it, and was
wondering why people were replying to it :|

I can't remember exactly in which release 'perfect English skills' were
added to Python runtime requirements, could you please refresh my memory?

I can't speak for Tyler (I assume your message was meant for him) but as
for myself:  I saw a glob of practically unreadable text and simply
passed it off as spam, concluding that any well-intentioned, moderately
intelligent human being wouldn't have intentionally posted such a
monstrosity and actually expected an intelligent response.

I'm guessing Tyler's message was to help you so that your messages don't
continue to be ignored by people who may otherwise be of assistance.



Sure the English wasn't the best but there was enough information in the 
post to identify the likely source of the problem - with a little help 
from my favourite search engine - and give the guy/gal some ideas of 
where to look for a solution.


If you were required to answer the question then asking the poster to 
phrase it better is going to help solve the issue faster but for a 
mailing list like this simply ignore it.


Stuart

--
http://mail.python.org/mailman/listinfo/python-list

Re: hmac module and key format

[Stuart Longland]

On Feb 21, 4:59 am, Peter Pearson  wrote:
> On Sun, 20 Feb 2011 04:01:20 -0800, Paul Rubin  
> wrote:
> > Stuart Longland  writes:
> >> What format does hmac require the key to be in?
>
> > It's an arbitrary string.  
>
> >     I have a key in hexadecimal, do I give it the hex?  Do I decode that
> >     to binary and give it that?  
>
> > Probably yes.  Do you have test vectors?  See if they work.
>
> Test case from http://www.faqs.org/rfcs/rfc2104.html:
[...]
> >>> hmac.hmac_md5( "Hi There", 16*"\x0b" )
>
> '\x92\x94rz68\xbb\x1c\x13\xf4\x8e\xf8\x15\x8b\xfc\x9d'

No worries, thanks to both you Peter and Paul, I'll give this a shot.
By the looks of things it is possible to just decode the hexadecimal
to a binary string and give it that.

I should perhaps elaborate on what I'm doing in case the specifics
make a difference.  I have a YubiKey which internally supports a
challenge-response mode based on HMAC-SHA1.  I've got a key, a sample
challenge and the sample output which is included in the python-yubico
module demos:

https://github.com/yubico/python-yubico

Before I worried about that though, I needed to have some kind of
understanding as to how the hmac module was used.  "Arbitrary string",
sounds to me like I give it something akin to a passphrase, and that
is hashed(?) to provide the symmetric key for the HMAC.  Wikipedia
seems to suggest it depends on the length of the key given, so if I
give it a string that's exactly 160-bits (for HMAC-SHA1) it'll use it
unmodified.  Would that be a correct assertion?
-- 
http://mail.python.org/mailman/listinfo/python-list

hmac module and key format

[Stuart Longland]

Hi,

Maybe I'm completely dense with regards to the hmac module and HMAC in
general, but I've searched and cannot find for the life of me the
answer to this very basic question.

What format does hmac require the key to be in?

I have a key in hexadecimal, do I give it the hex?  Do I decode that
to binary and give it that?  Do I try to figure out what passphrase
generated the hex and give it that instead?  Nowhere in the
documentation does it appear to mention what form the key must take,
or how you tell it what form it's in for it to "figure it out".  If
someone could enlighten me, I'd be most grateful.

Regards,
Stuart Longland
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: exceptions and unicode

[Stuart McGraw]

On 06/16/2010 03:51 PM, Thomas Jollans wrote:
> On 06/16/2010 10:10 PM, Stuart McGraw wrote:
>> Note that the exceptions may be anything (I just used IOError 
>> as an example) and are generated in bowels of an API that I 
>> can't/won't mess with.
> 
> Yeah, well, you'd have to special-case every single exception type that
> gets unicode arguments, as they probably all treat them in the same
> ungrateful way.

Unfortunate.  In general it does not seem possible to know what 
exceptions could be generated (they could be custom exceptions 
defined in the api) without examining all the code. 

Seems like I'll have to live with it or try some grotesque hack 
like looking for a repr-of-a-unicode-string-like text and converting
back to unicode.

Thanks.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: exceptions and unicode

[Stuart McGraw]

On 06/16/2010 03:53 PM, Martin v. Loewis wrote:
>> So how do I get what I want?
> 
> Submit a patch. You would have to explain why this is a bug fix and not 
> a new feature, as new features are not allowed anymore for 2.x.

Thanks.  Actually I have no idea if this is a bug or a feature 
(despite reading bug tracker issues 2517 and 6108, most of which
I did not understand) so I'm not in a position to argue either.  
What I think I'll do is note in my documentation that the unreadable
error messages are from Python and are only temporary for a couple 
years until the app can move to Python 3. 
-- 
http://mail.python.org/mailman/listinfo/python-list

exceptions and unicode

[Stuart McGraw]

I am having a problem with exceptions and unicode.

  try: open ('file.txt')
  except IOError, e: pass
  str (e)
  => "[Errno 2] No such file or directory: 'file.txt'"

which is fine but...

  try: open (u'フィイル.txt')
  except IOError, e: pass
  str (e)
  => "[Errno 2] No such file or directory: u'\\u30d5\\u30a3\\u30a4\\u30eb.txt'"

ok, I did ask for a str string so no reason I should expect a unicode
string, but then (in Python 2.6)...

  unicode (e)
  => u"(2, 'No such file or directory')"

i.e. no formatting at all, or in Python 2.6.5

  unicode (e)
  => u"[Errno 2] No such file or directory: u'\\u30d5\\u30a3\\u30a4\\u30eb.txt'"

i.e. the result is a unicode string in name only. :-(

What I was expecting (or at least hoping for) of course was 

  => u"[Errno 2] No such file or directory: '\u30d5\u30a3\u30a4\u30eb.txt'"

which when printed would produce something useful

  => [Errno 2] No such file or directory: 'フィイル.txt'

So how do I get what I want?  (Python 3.x is not an option.)
Note that the exceptions may be anything (I just used IOError 
as an example) and are generated in bowels of an API that I 
can't/won't mess with.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python books, literature etc

[Stuart Murray-Smith]

> Have a look at the Getting Started section of the wiki:
>
> http://wiki.python.org/moin/
>
> specially the PythonBooks section

Perfect! Exactly what I'm looking for :)

Thanks Gabriel!
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python books, literature etc

[Stuart Murray-Smith]

2010/1/6 J :
> A good point was brought up to me privately, and I agree completely,
> that the OP should re-state the request with a bit more specifics...
>
> Since the OP says he is at least familiar with Python, does he need
> info on beginner level books that are general purpose, or is he
> interested in resources that are more specific (e.g. geared toward web
> programming, mathematical analysis, data modeling, etc)
>
> My suggestions were meant just as an example of what I use in the
> course of learning something at the basic and intermediate level, once
> it goes beyond that, it's useful to know WHAT you intend to do so you
> can find the right resources to go in that direction.
>
> Just want to make sure I am not misunderstood or anything :-)
>
> For what it's worth, I also tend to collect technical books for some
> reason... My wife is just barely tolerant of my bookshelf full of
> things on various computer topics, astronomy, photography, radio and
> antenna theory and so forth ;-)  I just let her keep her shoe
> collection, and we have a quid pro quo.

Thanks J for your reply, much appreciated :)

Oops, vague OP, my bad.

Agreed, Google turns up myriad of topical books, and ESR's guide to
smart questions [1] helps set the pace of list culture. I subscribe to
various list servers, on one of them we gracefully accept that a
question like my OP is looking for opinion on a matter, which I now
understand would be a list's sub-culture?

Anyways, to rephrase, could someone kindly mention any of their
preferred Python books, websites, tutorials etc to help me get to an
intermediate/advanced level? Something that would help me add
functionality to Ubiquity, say.

Have a great day!

Stu@

[1] http://catb.org/~esr/faqs/smart-questions.html
-- 
http://mail.python.org/mailman/listinfo/python-list

Python books, literature etc

[Stuart Murray-Smith]

Greetings list

I can code in Python (strong beginner), and would like to read more
books and/or online resources.

Could someone please point out any good books, websites, tutorials etc
to help me get to the next level.

Your help && insight highly appreciated :)

Stuart
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PYTHONPATH issue with sibling package names

[Stuart Moffatt]

On Sep 10, 10:12 am, "Diez B. Roggisch"  wrote:
> Stuart Moffatt wrote:
> > Environment: Eclipse 3.4.2, Windows XP Pro SP2, Pydev 1.4.4, python
> > 2.6
>
> > When I work in eclipse with java, I like to break up my client and
> > server packages, like this:
>
> > client-project/src/org/me/client
>
> > server-project/src/org/me/api
> > server-project/src/org/me/dao
> > server-project/src/org/me/entity
> > server-project/src/org/me/
>
> > Then, when I need to call API code from the client, I make sure the
> > API src is on the path.
>
> > I am trying setup pydev projects to do the same thing, but running
> > into an ImportError because my client code can't see the server src,
> > even though it is on the path.
>
> > Specifically, I am trying to import an entity from the server code
> > into the client, like this:
>
> >    from org.me.entity import MyEntity
>
> > If I do this from any module in the server project it is fine (because
> > the src path is in the same eclipse project). But if I do it from
> > anywhere in the client code I get the ImportError
>
> > From what I can tell, python asks for the "closest" module path, which
> > is the current project. It finds org.me, but there is only the client
> > sub-package. The org.me.entity sibling is in another eclipse project,
> > but even though that path is on the PYTHONPATH, python stopped looking
> > after it found a similarly named parent package.
>
> > Is there a trusted way to make sure python looks through all paths for
> > sibling packages? Can I load 'org.me.client' from one path in
> > PYTHONPATH and 'org.me.*' from another path in PYTHONPATH? I imagine
> > if I try to force python to load the second package first that the
> > same thing will happen in reverse.
>
> The solution you are searching for is called "namespace packages", and you
> can read more about it here:
>
>  http://peak.telecommunity.com/DevCenter/setuptools#namespace-packages
>
> Do yourself a favor though, and don't use those several-steps-namespaces.
> This is Python, not Java - two levels at *most*, normally a
> project-namespace should be enough.
>
> Diez


Diez,

Thanks for the tips re: namespace packages. Yeah, I know this is
python, but for very large projects (or multiple projects for very
large clients) it is just more flexible to stick to the reverse-dot
java notation of "domain.organization.project.namespace".

All I had to do was make sure that top- and mid-level folders had an
__init__.py with this line:

__import__('pkg_resources').declare_namespace(__name__)

in it (and nothing else).

So, for my example, I had:

client-project/src/org/__init__.py
client-project/src/org/me/__init__.py
server-project/src/org/__init__.py
server-project/src/org/me/__init__.py

...all with the special namespace declaration line above.

Note that the lowest level folders in the package:

client-project/src/org/me/client
server-project/src/org/me/api
server-project/src/org/me/

...all have __init__.py files with actual module code, and therefore
do NOT have the namespace declaration line.

To expose the server code (org.me.api, org.me.dao, etc) I have a
setup.py in the server-project/src folder with:

from setuptools import setup
setup(name='myserver',
version='1.0',
namespace_packages = ['org','org.me'],
packages=['org.me.api','org.me.dao','org.me.entity'],
)

A similar setup.py can be created for the client library too. Notice
that the setup has every level (except the last) in the
namespace_packages list argument. This is how it walks down the tree
and builds the namespace properly.

With this setup file I can make an .egg or .zip distribution using:

  cd server-project/src
  python setup.py bdist

  or

  python setup.py bdist_egg

For my dev environment in eclipse I also wanted the convenience of
editing server code AND have the latest code on the path for my client
project (without rebuilding a dist). To do this I just:

  cd server-project/src
  python setup.py develop

...which puts a symlink (even does it properly in Windows) in my
python/lib/site-packages folder. Now in my client code I can do a
regular import, even though the code lives in another project, and
though portions of the code base are at different PYTHONPATHs:

  from org.me.entity import MyEntity # from server-project/src
  from org.me.api import MyAPI # from server-project/src
  from org.me.client import MyClient # from client-project/src

When my code is complete, the myserver-1.0-py2.6.egg and myclient-1.0-
py2.6.egg can be included in other python modules by loading compiled
source from inside the egg with:

  from

PYTHONPATH issue with sibling package names

[Stuart Moffatt]

Environment: Eclipse 3.4.2, Windows XP Pro SP2, Pydev 1.4.4, python
2.6

When I work in eclipse with java, I like to break up my client and
server packages, like this:

client-project/src/org/me/client

server-project/src/org/me/api
server-project/src/org/me/dao
server-project/src/org/me/entity
server-project/src/org/me/

Then, when I need to call API code from the client, I make sure the
API src is on the path.

I am trying setup pydev projects to do the same thing, but running
into an ImportError because my client code can't see the server src,
even though it is on the path.

Specifically, I am trying to import an entity from the server code
into the client, like this:

   from org.me.entity import MyEntity

If I do this from any module in the server project it is fine (because
the src path is in the same eclipse project). But if I do it from
anywhere in the client code I get the ImportError

>From what I can tell, python asks for the "closest" module path, which
is the current project. It finds org.me, but there is only the client
sub-package. The org.me.entity sibling is in another eclipse project,
but even though that path is on the PYTHONPATH, python stopped looking
after it found a similarly named parent package.

Is there a trusted way to make sure python looks through all paths for
sibling packages? Can I load 'org.me.client' from one path in
PYTHONPATH and 'org.me.*' from another path in PYTHONPATH? I imagine
if I try to force python to load the second package first that the
same thing will happen in reverse.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How to create a virtual serial port?

[Stuart Davenport]

On Apr 11, 6:56 pm, Grant Edwards  wrote:
> On 2009-04-11, Grant Edwards  wrote:
>
> > You can write a port redirector in user-space in MS-Windows,
> > but you can't in Linux/Unix.  On Unix systems you have to
> > write a kernel module that sits below the tty layer.
>
> Perhaps I should elucidate further.
>
> That's what the "pty" driver on Unix is: a kernel module that
> sits underneath the tty layer where the "normal" serial-port
> UART drivers sit.
>
> However, Unix pty drivers only handles a subset of the normal
> serial-port API.  [I'm not sure why pty drivers have never been
> "finished" so that they fully emulate a serial port, but it's
> been that way for 20+ years].
>
> If RouteBuddy doesn't try to do things like get/set modem
> control/status lines, then the OP might be able to use a pty.
>
> Each pty consists of two devices: a master end and a slave end.
> RouteBuddy would be told to use the slave end, and the OP would
> write a program that would transfer data between a network
> connection and the master end.
>
> A pty devices is what is used by programs like xterm to run
> programs like bash.  Xterm transfers data between a network
> connection and the master end of a pty.  Bash is connected to
> the slave end of that pty and "thinks" it's connected to a
> serial port.  Bash uses little of the serial port API, so it's
> happy with the limited API provided by a pty slave-end.
>
> --
> Grant

Grant - HERO! THIS EXACTLY WHAT I WANT TO DO...

I didn't realise people were still posting on this thread and have
spent the last 3 hours trying to get some example code to create a
pseudo TTY (pty) in Python, I get that I have to use the pty module,
but finding an example is like searching for... lets go festing ...an
easter egg in a forest ;)

You wouldn't have any pointers to initiating a PTY and how to write to
it? This is all I want to achieve.


My point in all this is actually that I ordered a USB GPS Receiver and
it wont arrive for another two weeks, being my impatient self, I am
writing an app for my iPhone to broadcast its GPS location over the
network to my laptop. I then want to get this data into the NMEA
format and push this data onto a PTY - this will in-effect replace the
USB GPS Receiver and the GPS software can read it :)

Cheers
Stu
--
http://mail.python.org/mailman/listinfo/python-list

Re: How to create a virtual serial port?

[Stuart Davenport]

On 11 Apr, 08:52, Scott David Daniels  wrote:
> Stuart Davenport wrote:
> > ... I'm on a OS X, python 2.5. Basically I will have a remote application
> > pushing data (GPS) over the network to a python application I have
> > running on my Mac, I want this python application to again push the
> > data on to a "virtual serial port". Then the GPS program I have
> > running on my MAC, RouteBuddy, can read the data from that serial port
> > as standard.
>
> > The only part I am concerned about here though, is if I can create a
> > serial port virtually and push data out of it?
>
> OK, this is a Max OS/X question.  Does "RouteBuddy" have any other way
> to get data except from the serial port?  Does the mac have a serial
> port?or are you talking "serial port over USB" (which would be an
> entirely different kettle of fish).  With that out of the wa, I'd try
> to ask some Mac guys if the ponying up is possible, and if so how to
> do it in _any_ language (likely they'll have an Objective C way).
> Then pop back over here (or even better a python-mac list, where
> they'll be familiar with how to do lots of stuff) and you can get
> help getting python to either do the same thing or talk to a bit
> of code that does the byte-transfers.
>
> --Scott David Daniels
> scott.dani...@acm.org

Sadly RouteBuddy cannot retrieve data by other means...

Scott, many thanks for the advice. Will try to find a more specific
forum of conversation.

Stu
--
http://mail.python.org/mailman/listinfo/python-list

Re: How to create a virtual serial port?

[Stuart Davenport]

On 10 Apr, 20:45, Scott David Daniels  wrote:
> Stuart Davenport wrote:
> > Hi,
>
> > I am trying to work out if its possible, to create a virtual serial
> > port with Python? Would anyone know how to go about this in code? Any
> > help would be greatly appreciated! :)
>
> > I have a had a google and the topics returned only seem to reflect
> > "reading" serial port data, particularly pySerial results...
>
> > Cheers
> > Stu
>
> Well, search for Smart Questions.  I have no idea what your OS is, nor
> what level of "virtual serial port" you want/need.  All anyone could
> do with your current request is guess what you need and try to steer you
> towards it.
>
> --Scott David Daniels
> scott.dani...@acm.org

Scott, totally fair point - a little candid, but fair...

I'm on a OS X, python 2.5. Basically I will have a remote application
pushing data (GPS) over the network to a python application I have
running on my Mac, I want this python application to again push the
data on to a "virtual serial port". Then the GPS program I have
running on my MAC, RouteBuddy, can read the data from that serial port
as standard.

The only part I am concerned about here though, is if I can create a
serial port virtually and push data out of it?

Hope that defines a little more clarity for yourself.
--
http://mail.python.org/mailman/listinfo/python-list

How to create a virtual serial port?

[Stuart Davenport]

Hi,

I am trying to work out if its possible, to create a virtual serial
port with Python? Would anyone know how to go about this in code? Any
help would be greatly appreciated! :)

I have a had a google and the topics returned only seem to reflect
"reading" serial port data, particularly pySerial results...

Cheers
Stu
--
http://mail.python.org/mailman/listinfo/python-list

Re: HTTPError... read the response body?

[Stuart Davenport]

On Mar 2, 11:50 pm, Wojtek Walczak  wrote:
> On Mon, 2 Mar 2009 14:29:12 -0800 (PST), Stuart Davenport wrote:
>
> Hi,
>
> > I am trying to connect to a web service but I am getting HTTP 400, I
> > am not too concerned about the HTTP error - but what I'd like to know
> > if there is anyway I can read the response body in the HTTP 400 or 500
> > case? Does the HTTPError allow this? or the urllib2 in anyway?
>
> HTTP error 400 means 'bad request', so it's almost certainly
> your mistake.
>
> >                         #url, body, headers
> >                         rq = urllib2.Request(args[1], args[3], headers)
>
> Is args[1] a valid URL? And are you sure that you want to send
> something to the web serwer? By specifying the second argument
> (args[3]) you're asking python to send HTTP "POST" request,
> not "GET".
>
> --
> Regards,
> Wojtek Walczak,http://tosh.pl/gminick/

Hi Wojtek,

Yes, the args[1] is a valid url (I've printed it out to check) and
thats correct, I do want to initiate a POST request by passing over
the SOAP Envelope in the request. All I'd like to know if there is
anyway to read the body of an HTTP Error, so for instance, if I were
to recieve a 404 - I'd like to see the content that the server returns
with a 404 error.
--
http://mail.python.org/mailman/listinfo/python-list

HTTPError... read the response body?

[Stuart Davenport]

Hi There,

I am trying to connect to a web service but I am getting HTTP 400, I
am not too concerned about the HTTP error - but what I'd like to know
if there is anyway I can read the response body in the HTTP 400 or 500
case? Does the HTTPError allow this? or the urllib2 in anyway?

This is what I have at the moment...

import sys, urllib2
import socket

class APIConnector:
def connect(*args):
length = len(args[2])
headers = {"Content-Length": length, "Content-Type":
"text/xml", "SOAPAction": "\"http://some.api.com/RemovedAction\""}

try:
#url, body, headers
rq = urllib2.Request(args[1], args[3],
headers)
rs = urllib2.urlopen(rq)

print rs.read()
except urllib2.HTTPError, e:
conn = e
print 'error', conn.code, 'message: ',
conn.msg, 'filename: ', conn.filename, 'headers: ', conn.hdrs, 'body:
', conn.read()

I thought that conn.read() may show the body, but it seems not... Can
anyone help please, or suggest another way around? I am using python
2.5 installed with MAC OS X Leopard.


Cheers
Stu
--
http://mail.python.org/mailman/listinfo/python-list

Installing python 2.6 on Vista

[GALLOWAY Stuart J (SVHM)]

I had python 2.5 on my computer and did a double install with 2.6 out of
curiosity.

Python 2.5 and idle had been working.

 

I was logged on as administrator (only user account on my computer) and
installed python2.6 for all users.

Python 2.6 idle did not work, and following installation nor did python
2.5

 

Reading the web suggestions I spent an evening trying combinations of
the following:

1.  Turning off UAC
2.  Uninstalling and reinstalling
3.  Installing for a single user
4.  Adding another admin account with a user
5.  Making a bat file and running the installer via the bat file as
administrator

 

Idle does still does not work on python 2.5 or 2.6

 

The only way I can get it to work is to create a new admin account with
a password and run it from this account.

The other (original) admin account cannot run idle, even if installed
for all users.

 

I do not want to do this because:

1. I have truly personal and physically secure computer and
I do not want to have to use a password

2. I want to use idle.

 

I accept that this may be essentially a VISTA problem.  However, I am a
fairly experienced user with many years of programming in python.

I less familiar with the ins and outs of windows especially vista but
part of the attraction of python is much easier than other environments
for doing most things.

If Python is this hard to getting going on windows for a reasonably
experienced user it should be regarded as BROKEN on windows.

 

I would strongly recommend that an official PORTABLE python for windows
should be a standard distribution so we can bypass all the windows
rubbish if we want to.  I am also going to try another distribution to
see if, at least, I can get python 2.5 back.

 

Not very happy,

 

Stuart


Disclaimer : The contents of this e-mail including any attachments are intended 
only for the person or entity to which this e-mail is addressed and may contain 
confidential, privileged and/or commercially sensitive material. If you are 
not, or believe you may not be, the intended recipient, please advise the 
sender immediately by return e-mail, delete this e-mail and destroy any copies.
__
This email has been scanned by the MessageLabs Email Security System.
For more information please visit http://www.messagelabs.com/email 
__--
http://mail.python.org/mailman/listinfo/python-list

Re: PyRun_String using my module in a def

[Stuart]

What command do you mean when you say "update main_dict with
dlfl_dict"?

I tried PyObject *rstring = PyRun_String( cmd, Py_file_input,
dlfl_dict, dlfl_dict );
This worked, but has the side effect of not allowing other commands
like "execfile"
I was able to type that before, but now it just says "name 'execfile'
is not defined"

Thanks for your help! :)

On Jul 4, 10:23 pm, [EMAIL PROTECTED] (Alex Martelli) wrote:
> Stuart <[EMAIL PROTECTED]> wrote:
>
>...
>
> > PyObject *rstring = PyRun_String( cmd, Py_file_input, main_dict,
> > dlfl_dict );
>
> You're passing difl_dict as the "locals" to PyRun_String -- but a
> function has its own locals, so it won't use those locals.  Just update
> main_dict with difl_dict (that's the equivalent of the "from difl import
> *" which you appear to desire) and use main_dict for both globals and
> locals in your PyRun_String call.
>
> Alex


-- 
http://mail.python.org/mailman/listinfo/python-list

need a variation algorithm for Lists in Dictionaries

[Marc Stuart]

Hi, I am trying to create a function, where I pass a dictionary with a
lits of strings, and try to return a
a list of strings, for all variations, any ideas ?
Thanks

def getAllVariants(someDict):
keys = someDict.keys()
for x in keys:
print len(someDict[x])


x = {1:['a','b'],2:['b','c'],3:['d','e','f','g']}
getAllVariants(x)

""" I need to get a list of strings that render all possible variants,
this is what my output should be based on the
x dictionary:
abd
abe
abf
acd
ace
acf
acg
bbd
bbe
bbf
bcd
bce
bcf
bcg

"""
-- 
http://mail.python.org/mailman/listinfo/python-list

PyRun_String using my module in a def

[Stuart]

I've written my own python modules with the C API, called dlfl. I've
now embedded a python interpreter into my Qt application. I am able to
execute multiline/singleline blocks and it has no problems remembering
definitions from one execute to the next. The problem arises when i
try and use a dlfl method inside a def block. For example

def x( ): print faces(0)

faces is a method in dlfl. If I were to just type "print faces(0)" it
would execute fine. However, this way when I type "x()", it just says
"global name 'faces' is not defined"

If I define x without any of my dlfl methods it works fine:

def x():
y = 2
print y+2

x()
# 4

Here are the relevant parts of my code:

PyObject *main = PyImport_AddModule("__main__");
PyObject *dlfl = PyImport_AddModule("dlfl");
PyObject* main_dict = PyModule_GetDict( main );
PyObject* dlfl_dict = PyModule_GetDict( dlfl );

PyObject *rstring = PyRun_String( cmd, Py_file_input, main_dict,
dlfl_dict );

rstring returns NULL and then PyErr_Fetch( &object, &data,
&traceback ); gets the traceback string as I mentioned above.

Hopefully, there is just a simple solution. Thanks :)

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pydoc with METH_VARGS

[Stuart]

I'm asking if there's some sort of commenting or input file or
something to customize the output pydoc generates. Thanks.

On Jun 23, 11:00 pm, 7stud <[EMAIL PROTECTED]> wrote:
> On Jun 23, 2:13 pm, Stuart <[EMAIL PROTECTED]> wrote:
>
> > With my Python extension module all the function definitions are with
> > METH_VARGS. The result being that pydoc, just puts "(...)" for the
> > argument list. Can I hand edit this to put the specific variable names
> > I want? With optional arguments in brackets or something?
>
> > Thanks.
>
> Are you asking whether you can change a text file that pydoc produces?

-- 
http://mail.python.org/mailman/listinfo/python-list

pydoc with METH_VARGS

[Stuart]

With my Python extension module all the function definitions are with
METH_VARGS. The result being that pydoc, just puts "(...)" for the
argument list. Can I hand edit this to put the specific variable names
I want? With optional arguments in brackets or something?

Thanks.

-- 
http://mail.python.org/mailman/listinfo/python-list

RE: Readline()

[Taylor, Stuart]

for anyone who gets this same problem i managed to resolve the issue
using two threads.
The main program spawned a subprocess and used readlines to read the
output. The process spawned was a script which executes a command using
a thread. The thread reads the output from this command using readline.
Within the main loop of the script there is a while loop which checks
when the last output from the command was and if the time elapsed is
greater than timeout the script prints to the stdout a string called
timeout.
In the main program the readline compares every output to see if it
matches this sting. if it does kill process :)
 
I'm sure this code could be implemented better but it works and got me
out of a hole
 
heres the thread script which contains the thread class and the time out
loop:
 
import datetime
import time
import threading
import subprocess
import os
import sys
class outputThread(threading.Thread):
"""
Thread is used to print the output of a command to the stdout
"""
def __init__(self,threadCommand, name="thread", *args, **kwargs):
threading.Thread.__init__(self, *args, **kwargs)
self.name   = name
self.killed = 0
self.counter= 0
self.command= threadCommand
self.retval = None
self.Set_Hit()
def run(self):
self.Proc   = subprocess.Popen(self.command, cwd =
os.getcwd(),stdin=subprocess.PIPE, stdout = subprocess.PIPE, stderr =
subprocess.STDOUT)
self.retval = None
self.Set_Hit()
while self.retval == None and not self.killed:   
line = self.Proc.stdout.readline()
print line.rstrip("\r\n")
sys.stdout.flush()
self.Set_Hit()
self.retval=self.Proc.poll()
def Set_Hit(self):
self.last_time = datetime.datetime.now()  
def Get_Hit(self):
return self.last_time

if __name__ == "__main__":
"""
This script is designed to put a timeout loop over a thread.
The intention of this thread is to perform a command and have a
timer loop over
checking for the last text ouput from the command.
If the loop exceeds the set time limit the command will stop reading
the output from the thread
"""
command = []
timeout = 5
maxCounter  = timeout
curCounter  = 0
strCounter  = 0
if len(sys.argv) > 2:
try:
timeout   = int(sys.argv[1])
except:
print "time out value is not a float"
sys.exit(1)
for argPos in range(2,len(sys.argv)):
print "arg = %s" % sys.argv[argPos]
command.append(sys.argv[argPos])
else:
print "not enough arguments"
sys.exit(1)
thread1 = outputThread(command,name="thread1")
thread1.start()
while True:
if not thread1.retval == None:
sys.exit(0)
#Get current time and when timeout should occur
currentTime = datetime.datetime.now()
elapsedTime = thread1.Get_Hit() +
datetime.timedelta(minutes=timeout)
#Check Elapsed time
if elapsedTime < currentTime:
print "timed out"
sys.stdout.flush()
break
time.sleep(1)



From: Taylor, Stuart [mailto:[EMAIL PROTECTED] 
Sent: 13 March 2007 09:50
To: Sick Monkey; Taylor, Stuart
Cc: python-list@python.org
Subject: RE: Readline()


i have a thread class which should read the output from the procedure
line by line and finish when the thread is set to kill:
 
class KillableThread(threading.Thread):
def __init__(self, name="thread", *args, **kwargs):
threading.Thread.__init__(self, *args, **kwargs)
self.name = name
self.killed = 0
def kill(self):
self.killed = 1
def run(self):
self.id = threading._get_ident()
self.Proc  =
subprocess.Popen(["python","-tt","output_file.py"], cwd =
os.getcwd(),stdin=subprocess.PIPE, stdout = subprocess.PIPE, stderr =
subprocess.STDOUT)
retval= None 
while retval == None and not self.killed:
line = self.Proc.stdout.readline()
retval=self.Proc.poll()
 
to replicate my problem the output_file.py requires a user input. i ask
for a user input as my thread will sit waiting on
self.Proc.stdout.readline(). This is the point i want to have a timeout
or be able to end my thread as there is no output from the output_file. 
 
The main approch i cannot implement is to be able to kill the thread as
it remains hung on readlines()
 



From: Sick Monkey [mailto:[EMAIL PROTECTED] 
Sent: 13 March 2007 01:51
To: Taylor, Stuart
Cc: python-list@python.org
Subject: Re: Readlin

RE: Readline()

[Taylor, Stuart]

i have a thread class which should read the output from the procedure
line by line and finish when the thread is set to kill:
 
class KillableThread(threading.Thread):
def __init__(self, name="thread", *args, **kwargs):
threading.Thread.__init__(self, *args, **kwargs)
self.name = name
self.killed = 0
def kill(self):
self.killed = 1
def run(self):
self.id = threading._get_ident()
self.Proc  =
subprocess.Popen(["python","-tt","output_file.py"], cwd =
os.getcwd(),stdin=subprocess.PIPE, stdout = subprocess.PIPE, stderr =
subprocess.STDOUT)
retval= None 
while retval == None and not self.killed:
line = self.Proc.stdout.readline()
retval=self.Proc.poll()
 
to replicate my problem the output_file.py requires a user input. i ask
for a user input as my thread will sit waiting on
self.Proc.stdout.readline(). This is the point i want to have a timeout
or be able to end my thread as there is no output from the output_file. 
 
The main approch i cannot implement is to be able to kill the thread as
it remains hung on readlines()
 



From: Sick Monkey [mailto:[EMAIL PROTECTED] 
Sent: 13 March 2007 01:51
To: Taylor, Stuart
Cc: python-list@python.org
Subject: Re: Readline()


Maybe if you show us your code we can better assist you.  

But maybe you can use a global variable or a try-catch method to keep
threads from staying alive and help better control your code.  


On 3/12/07, Taylor, Stuart <[EMAIL PROTECTED]> wrote: 

I have been working on running an external process using
subprocess.popen for a few days. 
The process is ran over the network to another machine. 
One thing I have found is that if I perform readline() on the
stdout it will hang if the process loses connection. 

I have tried a few things in a test example: one is to use
stdin.write then stdin.flush() which send a string that readline() will
read and it ends correctly but doesn't work on the project I need it to
work on. Another is to try using threads and ignoar the thread when  the
process has lost connection but this still leaves the open thread alive
even when the main app has finished. 

I am relatively new to python and I may be making a fundemantal
mistake in my implementation, can anyone please inform me of whether
sticking with subprocess.popen and readline() is the correct procedure
for this sort of task? 

And if anyone can point me in the correct implementation of this
problem I would be gratefull. 


Thank you 

Stuart 


--
http://mail.python.org/mailman/listinfo/python-list



-- 
http://mail.python.org/mailman/listinfo/python-list

Readline()

[Taylor, Stuart]

I have been working on running an external process using
subprocess.popen for a few days.
The process is ran over the network to another machine.
One thing I have found is that if I perform readline() on the stdout it
will hang if the process loses connection.

I have tried a few things in a test example: one is to use stdin.write
then stdin.flush() which send a string that readline() will read and it
ends correctly but doesn't work on the project I need it to work on.
Another is to try using threads and ignoar the thread when  the process
has lost connection but this still leaves the open thread alive even
when the main app has finished.

I am relatively new to python and I may be making a fundemantal mistake
in my implementation, can anyone please inform me of whether sticking
with subprocess.popen and readline() is the correct procedure for this
sort of task?

And if anyone can point me in the correct implementation of this problem
I would be gratefull.


Thank you

Stuart
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: pytz2007c error

[Stuart Bishop]

looping wrote:

On Mar 6, 9:51 am, "looping" <[EMAIL PROTECTED]> wrote:

Hi,
Why this error ?


from pytz import timezone
eastern = timezone('US/Eastern')

Traceback (most recent call last):
  File "", line 1, in 
  File "C:\Python25\lib\site-packages\pytz-2007c-py2.5.egg\pytz
\__init__.py", line 93, in timezone
  File "C:\Python25\lib\site-packages\pytz-2007c-py2.5.egg\pytz
\tzfile.py", line 33, in build_tzinfo
for trans in data[:timecnt]]
  File "C:\Python25\lib\site-packages\pytz-2007c-py2.5.egg\pytz
\tzinfo.py", line 27, in memorized_datetime
dt = datetime.utcfromtimestamp(seconds)
ValueError: timestamp out of range for platform localtime()/gmtime()
function

I'm running python 2.5 on WinXP French with the egg from CheeseShop.

Same error with all timezone (like timezone('Europe/Zurich'),
timezone('Europe/Amsterdam'), ...) except a few one:>>> print timezone('UTC')

UTC

Is this a pytz problem or something I didn't understand ?


OK, the error come from datetime.utcfromtimestamp that doesn't support
negative value.
pytz try to pass -1633280400 to this function.

Is this a problem from Windows ?


Looks like it. I've opened a bug.

https://bugs.beta.launchpad.net/pytz/+bug/90096

Current workaround is to use an earlier release. This will be fixed as pytz 
needs to remain cross platform.


--
Stuart Bishop <[EMAIL PROTECTED]>
http://www.stuartbishop.net/



signature.asc
Description: OpenPGP digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Is Python Right for Me?

[Stuart D. Gathman]

On Fri, 02 Feb 2007 15:09:20 -0500, Mister Newbie wrote:

> I want to make small, 2D games. I have no programming experience. Is
> Python a good choice?

Definitely.  I teach a class for 7th to 12th grade where I use this
tutorial to introduce programming:

http://www.livewires.org.uk/python/
http://www.livewires.org.uk/python/pdfsheets.html

As an adult, just skip rapidly through the elementary material.  The final
module (Games sheets) walks you through creating 3 2D games with pygame!

-- 
  Stuart D. Gathman <[EMAIL PROTECTED]>
Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flamis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Vim scripting with python

[Stuart D. Gathman]

On Sat, 03 Feb 2007 05:02:54 -0800, Tool69 wrote:

> Does anyone have any advice, and more genraly how to script Vim with
> Python ?

:py import sys
:py print sys.version
:help :py

> I know I can put some python functions inside my vimrc file like
> this :
> 
> function! My_function()
> python << EOF
> import vim, string
> ...blablabla
> EOF
> endfunction
> 
> but I would like to use external ".py" files.

:py import myfile

Use :py inside your vimrc - don't run python externally.
-- 
  Stuart D. Gathman <[EMAIL PROTECTED]>
Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flamis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Debugging SocketServer.ThreadingTCPServer

[Stuart D. Gathman]

On Tue, 16 Jan 2007 09:11:38 -0500, Jean-Paul Calderone wrote:

> On Tue, 16 Jan 2007 00:23:35 -0500, "Stuart D. Gathman"
> <[EMAIL PROTECTED]> wrote:
>>I have a ThreadingTCPServer application (pygossip, part of
>>http://sourceforge.net/projects/pymilter).  It mostly runs well, but
>>occasionally goes into a loop.  How can I get a stack trace of running
>>threads to figure out where the loop is?  Is there some equivalent of
>>sending SIGQUIT to Java to get a thread dump?  If needed, I can import
>>pdb and set options at startup, but there needs to be some external way
>>of triggering the dump since I can't reproduce it at will.
> 
> Grab the gdbinit out of Python SVN Misc/ directory.  Apply this patch:
> 
>   http://jcalderone.livejournal.com/28224.html
> 
> Attach to the process using gdb.  Make sure you have debugging symbols
> in your build of Python.  Run 'thread apply all pystack'.

Did this.  gdb displays main thread fine (waiting on accept(), duh).  But
gdb goes into a loop displaying the first worker thread.  There are no
extension modules other than the batteries included ones. In this
application, I believe, only _socket.  (I.e. a pure python server.)

I will try for a C stack trace next time it loops.

Also, the looping server needs kill -9.  SIGTERM and SIGINT won't stop it.
And after it dies with SIGKILL, the port is still in use for 5 minutes or
so (and the server cannot be restarted).

This is really making me appreciate Java.

-- 
  Stuart D. Gathman <[EMAIL PROTECTED]>
Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flamis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

-- 
http://mail.python.org/mailman/listinfo/python-list

doctest problem with null byte

[Stuart D. Gathman]

I am trying to create a doctest test case for the following:

def quote_value(s):
"""Quote the value for a key-value pair in Received-SPF header field
if needed.  No quoting needed for a dot-atom value.

>>> quote_value(r'abc\def')
'"abcdef"'
>>> quote_value('abc..def')
'"abc..def"'
>>> quote_value('')
'""'
>>> quote_value('-all\x00')
'"-allx00"'
...

However, doctest does *not* like the null byte in the example (yes, this
happens with real life input):
**
File "/home/stuart/pyspf/spf.py", line 1453, in spf.quote_value
Failed example:
quote_value('-all')
Exception raised:
Traceback (most recent call last):
  File "/var/tmp/python2.4-2.4.4c1-root/usr/lib/python2.4/doctest.py",
line 1248, in __run
compileflags, 1) in test.globs
TypeError: compile() expected string without null bytes
**

How can I construct a test cast for this?

-- 
  Stuart D. Gathman <[EMAIL PROTECTED]>
Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flamis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Best way to document Python code...

[Stuart D. Gathman]

On Mon, 22 Jan 2007 17:35:18 -0500, Stuart D. Gathman wrote:

> The HTML generated by pydoc doesn't link to standard modules properly. 
> They are generated as relative links.  So it can't be used without
> modification for generating docs for a web page about a python package. 
> 
> I'm struggling with the same issue.  Coding Python is so much easier than
> Java.  However documenting Java is so much easier than Python.  Just
> include doc comments, run javadoc, and voila!

Wow!  I just tried epydoc, and it is every bit as easy as javadoc and
with similar output. Too bad it isn't standard.  But the comments and
docstrings it parses work fine with pydoc also.

-- 
  Stuart D. Gathman <[EMAIL PROTECTED]>
Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flamis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Best way to document Python code...

[Stuart D. Gathman]

On Mon, 22 Jan 2007 20:40:57 +, Adonis Vargas wrote:

> But a quick look at pydoc (not to be confused with epydoc) 
> which is part of the standard library allows you to generate 
> documentation in HTML format, and/or serve it over web with its built-in 
> HTTP server.
> 
> pydoc: http://docs.python.org/lib/module-pydoc.html
> 
> Hope this helps.
> 
> Adonis

The HTML generated by pydoc doesn't link to standard modules properly. 
They are generated as relative links.  So it can't be used without
modification for generating docs for a web page about a python package. 

I'm struggling with the same issue.  Coding Python is so much easier than
Java.  However documenting Java is so much easier than Python.  Just
include doc comments, run javadoc, and voila!

-- 
  Stuart D. Gathman <[EMAIL PROTECTED]>
Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flamis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

-- 
http://mail.python.org/mailman/listinfo/python-list

Debugging SocketServer.ThreadingTCPServer

[Stuart D. Gathman]

I have a ThreadingTCPServer application (pygossip, part of 
http://sourceforge.net/projects/pymilter).  It mostly runs well, but
occasionally goes into a loop.  How can I get a stack trace of running
threads to figure out where the loop is?  Is there some equivalent of
sending SIGQUIT to Java to get a thread dump?  If needed, I can import pdb
and set options at startup, but there needs to be some external way of
triggering the dump since I can't reproduce it at will.

-- 
  Stuart D. Gathman <[EMAIL PROTECTED]>
Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flamis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: I'm looking for a pythonic red-black tree...

[Stuart D. Gathman]

On Fri, 15 Dec 2006 01:20:34 +, Just Another Victim of the Ambient
Morality wrote:

> I need a red-black tree in Python and I was wondering if there was one 
> built in or if there's a good implementation out there.  Something that, 
> lets face it, does whatever the C++ std::map<> allows you to do...

Are you really looking specifically for a red-black tree, or do you want a
container where iterators return values in sorted order?  For instance, my
favorite sorted container is the skip-list:

 * inherently thread safe even during container updates
 * updates as fast as searches - significantly faster than red-black tree
 * search as fast as trees on average - but probablistic (like hashtable)
 * sequential access as fast as a linked list
 * Uses 33% less memory than binary trees (like red-black tree)
 * in general, performs like a hashtable, but sorted

Java example: http://bmsi.com/java/SkipList.java

In Python, the performance of a pure Python container will be
disappointing unless it leverages a native container.  For many
applications, you can use a dict and sort when iterating (using heapq to
amortize the sorting).

If I ever get the time, I would seriously consider trying to modify Python
to replace the built-in dict with a skip-list algorithm and compare the
performance.  Failing that, an extension module implementing a sorted
container of some description would be useful.

-- 
  Stuart D. Gathman <[EMAIL PROTECTED]>
Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flamis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

-- 
http://mail.python.org/mailman/listinfo/python-list

Package vs. module

[Stuart D. Gathman]

On Mon, 11 Dec 2006 20:59:27 -0300, Gabriel Genellina wrote:

> The above code *almost* works, but DNSLookup is a local name inside 
> the function. Use the global statement.
> As an example, see how getpass.py (in the standard library) manages 
> the various getpass implementations.

Ok, I have a working package:

spf/
  __init__.py
  pyspf.py
  pydns.py
  dnspython.py

__init__.py:
from pyspf import *
from pyspf import __author__,__email__,__version__ 

def set_dns_driver(f):
  global DNSLookup
  DNSLookup = f
  pyspf.DNSLookup = f

def DNSLookup(name,qtype,strict=True):
  import pydns
  return DNSLookup(name,qtype,strict)

set_dns_driver(DNSLookup)

Importing a driver module activates that driver.
For instance, in pydns.py:

import DNS# http://pydns.sourceforge.net
import spf

...
def DNSLookup(...):
  ...

spf.set_dns_driver(DNSLookup)

NOW, this is all very nice and modular. BUT, the original module was a
single file, which could be run as a script as well as imported as a
module. The script features provided useful command line functionality.
(Using if __name__ == '__main__':).  Now that 'spf' is a package, the
command line feature is gone!  Even using -m, I get:

python2.4 -m spf
python2.4: module spf has no associated file

Looking at getpass.py as advised, I see they put all the drivers in the
module.  I could do that with spf.py, I suppose.  But I like how with the
package, the driver code is not loaded unless needed.

One other idea I had was an arrangement like this:

SPF/
  pydns.py
  dnspython.py

spf.py

This would keep the module as a single file usable from the command line,
but still make driver available as separately loaded modules.

So which of the three options,

1) single file module with all drivers, ala getpass
2) package that cannot be run directly from command line
3) single file module with associated driver package

is the most pythonic?


-- 
  Stuart D. Gathman <[EMAIL PROTECTED]>
Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flamis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Automatic debugging of copy by reference errors?

[Stuart D. Gathman]

On Sat, 09 Dec 2006 05:58:22 -0800, Niels L Ellegaard wrote:

> I wanted  a each object to know whether or not it was being referred to
> by a living object, and I wanted to warn the user whenever he tried to
> change an object that was being refered to by a living object.  As far
> as I can see the garbage collector module would allow to do some of
> this, but one would still have to edit the assignment operators of each
> of the standard data structures:

I think what you want is a namespace that requires each object to have
exactly one reference - the namespace.  Of course, additional references
will be created during evaluation of expressions.  So the best you can do
is provide a function that checks reference counts for a namespace when
called, and warns about objects with multiple references.  If that could
be called for every statement (i.e. not during expression evaluation -
something like C language "sequence points"), it would probably catch the
type of error you are looking for. Checking such a thing efficiently would
require deep changes to the interpreter.

The better approach is to revel in the ease with which data can be
referenced rather than copied.  I'm not sure it's worth turning python
into fortran - even for selected namespaces.

-- 
  Stuart D. Gathman <[EMAIL PROTECTED]>
Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flamis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Driver selection

[Stuart D. Gathman]

On Fri, 08 Dec 2006 21:35:41 -0800, Gabriel Genellina wrote:

> On 9 dic, 00:53, "Stuart D. Gathman" <[EMAIL PROTECTED]> wrote:

>> Or you can modify the source to "from drivermodule import DNSLookup".
>>
>> What is the friendliest way to make this configurable?  Currently, users
>> are modifying the source to supply the desired driver.  Yuck.  I would
>> like to supply several drivers, and have a simple way to select one at
>> installation or run time.
> 
> You can store the user's choice in a configuration file (like the ones
> supported by ConfigParser).
> Then you can put all the logic to select and import the right function
> in a separate module, and export the DNSLookup name; make all the other
> parts of the application say "from mymodule import DNSLookup"

pyspf is a library, not an application.  It would be nasty to make it have
a config file.  We already have "from pydns_driver import DNSLookup" in
the pyspf module.  If only users could import *for* a module from
*outside* the module.  I suppose you can do something like this:

app.py:

import spf
# select dnspython driver for spf module
from spf.dnspython_driver import DNSLookup
spf.DNSLookup = DNSLookup
del DNSLookup

...

That is ugly.  I'm looking for better ideas.  Is there a clean way to make
a setdriver function?  Used like this, say:

app.py:

import spf
spf.set_driver('dnspython')
...

Can a function replace itself?  For instance, in spf.py, could DNSLookup
do this to provide a default:

def set_driver(d):
  if d == 'pydns':
from pydns_driver import DNSLookup
  elif d == 'dnspython':
from dnspython_driver import DNSLookup
  else: raise Exception('Unknown DNS driver')

def DNSLookup(name,t):
  from pydns_driver import DNSLookup
  return DNSLookup(name,t)

-- 
  Stuart D. Gathman <[EMAIL PROTECTED]>
Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flamis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

-- 
http://mail.python.org/mailman/listinfo/python-list

Driver selection

[Stuart D. Gathman]

The pyspf package [http://cheeseshop.python.org/pypi/pyspf/] can use
either pydns, or dnspython.  The pyspf module has a simple driver
function, DNSLookup(), that defaults to the pydns version.  It can be
assigned to a dnspython version, or to a test driver for in memory DNS. 
Or you can modify the source to "from drivermodule import DNSLookup".

What is the friendliest way to make this configurable?  Currently, users
are modifying the source to supply the desired driver.  Yuck.  I would
like to supply several drivers, and have a simple way to select one at
installation or run time.

-- 
  Stuart D. Gathman <[EMAIL PROTECTED]>
Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flamis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: NFS server

[Stuart D. Gathman]

On Fri, 24 Nov 2006 05:00:53 -0800, srj wrote:

> i wish to develop an NFS server usin python from scratch( some wise guy
> told me i'ts easy!).
> can i get any kinda tutorial for this??
> 
> any suggestions on how 2 begin?

NFS is an RPC based protocol.  The first step is to be able to do
SunRCP/ONCRPC. Python has an 'xdrlib' which is how parameters are
marshalled, unmarshalled in sunrpc.  If allowed under "from scratch", you
could wrap the C rpc lib for python - they handle retries and other low
level stuff.  You could look "Remote Tea" for Java, and translate to
python to get a pure python oncrpc lib.  Or you could look for such a
package already written (a quick search didn't reveal any).

Once you have rpc, then it is "just" a matter of having your python server
implement the set of calls specified for NFS.  BTW, apparently python was
used for quickly building test rigs while developing NFS v4.

Having a framework for python NFS server could be useful - think custom
filesystem.  Although a python binding for fuse + C NFS server would
be more general (use locally as well as remotely).

-- 
  Stuart D. Gathman <[EMAIL PROTECTED]>
Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flamis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

-- 
http://mail.python.org/mailman/listinfo/python-list

File upload from client application (non-form based upload)

[stuart]

Hi

I'm trying to write a Python script to receive and save a file on a web
server that has been POST'ed from a client application.

In essence, this is similar to handling a file upload from an HTML
form. However, I can't use:

form = cgi.FieldStorage()
fileitem = form['file']

since the file is not coming from a form, and hence I don't have a form
field called 'file'.

I have working server-side code in PHP to do this (error handling
removed):

$file = "./test.jpg";
$file_handle = fopen($file,"w");
$mydata = file_get_contents("php://input");
fwrite($file_handle, $mydata);
fclose($file_handle);

What I need is a Python equivalent of the the above PHP script. The
content-type in the POST header is currently set to
"application/octet-stream" which works fine with the php code above.

Any help, advise, pointers, sample code would be hugely welcome,

Many thanks in advance,

Stuart

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: psycopg2 faster way to retrieve last x records

[Stuart Bishop]

Luis P. Mendes wrote:
> Hi,
> 
> I would like to know if there is a better way to do what I'm already doing
> as stated in the following example, when using psycopg2 with PostgresQL.
> ...
> nr_bars_before = 200
> tabela = 'seconds'
> sqlString = "SELECT * FROM " + tabela + " ORDER BY tempounix;"
> curs = self.conn.cursor()
> curs.execute(sqlString)
> try:
>   while 1:
>   curs.scroll(1,mode='relative')
> except: pass
> curs.scroll(-int(math.fabs(nr_bars_before)),mode='relative')
> row = curs.fetchone()
> curs.close()
> ...
> 
> What I need is to get the last 200
> records from the table, each couple minutes.  As stated,
> what I do is to go all the way through the table records until the end,
> then going back 200 in order to select all of them (those 200) forward
> down to the last one.
> 
> But it takes a lot of time to do it.  I mean some seconds.  And it brings
> some 'heavy' work on disk.  The table 'seconds' has 54+ lines right
> now. 

The following SQL statement will return the last 200 rows in reverse order:

SELECT * FROM seconds ORDER BY tempounix DESC LIMIT 200

This will only send 200 rows from the server to the client (your existing
approach will send all of the rows). Also, if you have an index on tempounix
it will be really fast.


If you really need the results in tempounix order, then:

SELECT * FROM (
SELECT * FROM seconds ORDER BY tempounix DESC LIMIT 200
) AS whatever
ORDER BY tempounix;


-- 
Stuart Bishop <[EMAIL PROTECTED]>
http://www.stuartbishop.net/



signature.asc
Description: OpenPGP digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: does raw_input() return unicode?

[Stuart McGraw]

"Martin v. Löwis" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED]
> Stuart McGraw schrieb:
> > So, does raw_input() ever return unicode objects and if
> > so, under what conditions?
>
> At the moment, it only returns unicode objects when invoked
> in the IDLE shell, and only if the character entered cannot
> be represented in the locale's charset.

Thanks for the answer.

Also, if anyone has a solution for Duncan Booth's attempt
to wrap stdin, I would find it very useful too!

"Duncan Booth" <[EMAIL PROTECTED]> wrote:
> "Stuart McGraw" <[EMAIL PROTECTED]> wrote:
>
> > So, does raw_input() ever return unicode objects and if
> > so, under what conditions?
> >
> It returns unicode if reading from sys.stdin returns unicode.
>
> Unfortunately, I can't tell you how to make sys.stdin return unicode for
> use with raw_input. I tried what I thought should work and as you can see
> it messed up the buffering on stdin. Does anyone else know how to wrap
> sys.stdin so it returns unicode but is still unbuffered?
>
> Python 2.5 (r25:51908, Sep 19 2006, 09:52:17) [MSC v.1310 32 bit (Intel)]
> on win32
> Type "help", "copyright", "credits" or "license" for more information.
> >>> import sys, codecs
> >>> sys.stdin.encoding
> 'cp437'
> >>> sys.stdin = codecs.getreader(sys.stdin.encoding)(sys.stdin)
> >>> raw_input()
> hello world
> still going?
> ^Z
> ^Z
> u'hello world'
> >>>

-- 
http://mail.python.org/mailman/listinfo/python-list

does raw_input() return unicode?

[Stuart McGraw]

In the announcement for Python-2.3 
http://groups.google.com/group/comp.lang.python/msg/287e94d9fe25388d?hl=en
it says "raw_input(): can now return Unicode objects".

But I didn't see anything about this in Andrew Kuchling's
"2.3 What's New", nor does the current python docs for 
raw_input() say anything about this.  A test on a MS 
Windows system with a cp932 (japanese) default locale 
shows the object returned by raw_input() is a str() object 
containing cp932 encoded text.  This remained true even
when I set Python's default encoding to cp932 (in 
sitecustomize.py).

So, does raw_input() ever return unicode objects and if
so, under what conditions?

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: does anybody earn a living programming in python?

[Stuart Bishop]

walterbyrd wrote:

> My research of this subject was very limited, just looked at the major
> job boards, and compared demand for python developers to demand for
> other languages, such as java, c++, visual basic, or php. An
> unscientific test, I realize. But, it's not easy to get good data on
> such a subject: everybody has an agenda, it seems.
> 
> I don't count jobs where python is an "also ran" skill. For example,
> sys-admin jobs where python is listed with a dozen other skills.

I think you are getting skewed results. The companies I have been involved
in that use Python tend to be after top-tier (or as close as they can
afford) programmers rather than the sort of applications you get by the
truckload if you advertise a job on 'the major job boards'. You get a much
better quality of applicant if you target niche boards (such as
http://www.python.org/community/jobs/), use word of mouth, or posts to
highly targeted mailing lists like the ones run by your companies local
users groups or direct contact with visible community members with the
skillsets you need. The results get further skewed on 'the major job boards'
when one real job results in 100 job notices from 100 different recruitment
companies.

My personal experience is that there is a shortage of good Python
programmers. In Melbourne, Australia for example there is a continual need
for about 2 more - one Python shop there just hires clueful developers and
makes their first task 'learn Python'. We generally have a few positions
open at any particular moment too - http://www.ubuntu.com/employment (but we
are picky - it is hard to hand hold when the hand is several time zones away).

On a slight tangent, one of the appealing things about Python (and other
'non-mainstream' languages) in the past has been that the people who knew
Python also tended to be the people you wanted to employ - they generally
had experience in other languages but moved onto something they perceived as
'better' either at work or home. It indicated a level of care and pride
about their profession rather than someone who just treated cutting code as
a day job. That might be changing now that Python is becoming more visible
on peoples radar.

-- 
Stuart Bishop <[EMAIL PROTECTED]>
http://www.stuartbishop.net/



signature.asc
Description: OpenPGP digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: PostgreSQL, psycopg2 and OID-less tables

[Stuart Bishop]

Frank Millman wrote:

> I used to use 'select lastval()', but I hit a problem. If I insert a
> row into table A, I want the id of the row inserted. If it is a complex
> insert, which triggers inserts into various other tables, some of which
> may also be auto-incrementing, lastval() returns the id of the row last
> inserted into any auto-incrementing table.
> 
> I therefore use the following -
> cur.execute("select currval('%s_%s_Seq')" % (tableid, columnid)
> 
> where tableid is the specific table I want (in this example, table A),
> and columnid is the column specified as the auto-incrementing one.

If you are using a modern PostgreSQL (8.1 for sure, maybe 8.0), this is
better spelt:

cur.execute("SELECT currval(pg_get_serial_sequence(%s, %s))" % (
tableid, columnid))

(Assuming of course your table name and column name don't contain odd
characters like = or ' or ., in which case you need to properly quote them).

The reason it is better is that in odd cases the sequence name will not
always be %(tableid)s_%(columnid)s_seq, such as after you have renamed a table.

-- 
Stuart Bishop <[EMAIL PROTECTED]>
http://www.stuartbishop.net/



signature.asc
Description: OpenPGP digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: A critique of cgi.escape

[Stuart Bishop]

Jon Ribbens wrote:
> In article <[EMAIL PROTECTED]>, Georg Brandl wrote:
>>> I'm sorry, that's not good enough. How, precisely, would it break
>>> "existing code"? Can you come up with an example, or even an
>>> explanation of how it *could* break existing code?
>> Is that so hard to see? If cgi.escape replaced "'" with an entity reference,
>> code that expects it not to do so would break.
> 
> Sorry, that's still not good enough. Why would any code expect such a
> thing?

Plenty of test suites for a start. A non-backwards compatible change such as
being suggested can create a huge maintenance burden on lots of people.
People also use that function to escape non-HTML too - if they are using it
as documented, and it produces the correct results for them, great. Note
that the documentation doesn't say that input has to be HTML, nor that
output must be used as HTML. It just describes the transformation that it
does clearly and unambiguously and can quite happily be used for generating
 quoted text for use in, say, XML documents. Also, because Python has a
conservative policy on backwards incompatible changes, you are protected
from some wanker going and changing the HTML safe mappings arbitrarily, say
using numerical entity references instead of >, < and &. This
policy allows large software projects to be developed in Python and
maintained with less pain than if they were written in languages with a less
conservative policy.

If you want to improve the situation, join the WEB-SIG to help design new
and improved APIs so that the existing ones like the ancient cgi module can
be deprecated. Or maybe just some helpers can be added to the existing
htmllib module? There are better approaches than making non-backwards
compatible changes to functions people have been relying on since Python 1.5.


-- 
Stuart Bishop <[EMAIL PROTECTED]>
http://www.stuartbishop.net/



signature.asc
Description: OpenPGP digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: elementtree: line numbers and iterparse

[Stuart McGraw]

"Fredrik Lundh" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED]
> Stuart McGraw wrote:
> > Now I find i need to get and save the input file line 
> > number of each node.  Googling turned up a way 
> > to do it by subclassing FancyTreeBuilder,
> > (http://groups.google.com/group/comp.lang.python/msg/45f5313409553b4b?hl=en&;)
> > but that tries to read everything at once.
> > 
> > Is there a way to do something similiar with iterparse()?
> 
> something like this could work:
> ...snip...

Indeed it does.  Many thanks!

-- 
http://mail.python.org/mailman/listinfo/python-list

elementtree: line numbers and iterparse

[Stuart McGraw]

I have a broad (~200K nodes) but shallow xml file
I want to parse with Elementtree.  There are too many 
nodes to read into memory simultaneously so I use
iterparse() to process each node sequentially.

Now I find i need to get and save the input file line 
number of each node.  Googling turned up a way 
to do it by subclassing FancyTreeBuilder,
(http://groups.google.com/group/comp.lang.python/msg/45f5313409553b4b?hl=en&;)
but that tries to read everything at once.

Is there a way to do something similiar with iterparse()?

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: smtplib timeout

[Stuart D. Gathman]

On Tue, 25 Jul 2006 09:21:40 -0700, Alan Kennedy wrote:

> [Stuart D. Gathman]
>> I need to set a timelimit for the operation of
>> smtplib.sendmail.  It has to be thread based, because pymilter uses
>> libmilter which is thread based.
> 
> Have you tried setting a default socket timeout, which applies to all
> socket operations?

Does this apply to all threads, is it inherited when creating threads, or
does each thread need to specify it separately?

-- 
  Stuart D. Gathman <[EMAIL PROTECTED]>
Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flamis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

-- 
http://mail.python.org/mailman/listinfo/python-list

smtplib timeout

[Stuart D. Gathman]

I am doing SMTP callbacks in a Python milter
(http://pymilter.sourceforge.net) using the smtplib module.  For some
spammer MXes, it takes days (!) before smtplib.sendmail will return. 
Since the spammer connects to us every few seconds, this quickly leads to
a problem :-)

I need to set a timelimit for the operation of
smtplib.sendmail.  It has to be thread based, because pymilter uses
libmilter which is thread based.  There are some cookbook recipies which
run a function in a new thread and call Thread.join(timeout).  This
doesn't help, because although the calling thread gets a nice timeout
exception, the thread running the function continues to run.  In fact, the
problem is worse, because even more threads are created.

-- 
  Stuart D. Gathman <[EMAIL PROTECTED]>
Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flamis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Help beautify ugly heuristic code

[Stuart D. Gathman]

On Thu, 09 Dec 2004 00:01:36 -0800, Lonnie Princehouse wrote:
 
> I believe you can still do this with only compiling a regex once and
> then performing a few substitutions on the hostname.

That is a interesting idea.  Convert ip matches to fixed patterns, and
*then* match the regex.  I think I would convert hex matches to the same
pattern as decimal (and roman numeral).  How would you handle zero fill?

1.2.3.4 001002003004foo.isp.com

An idea I had last night is to precompile 254 regexes - one for each of
the possible last ip bytes.  However, your idea is cleaner - except, how
would it handle ip bytes that are the same: 1.2.2.2

Mitja has proposed a scoring system reminiscent of SpamAssassin.

This gives me a few things to try.

-- 
  Stuart D. Gathman <[EMAIL PROTECTED]>
Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flamis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

-- 
http://mail.python.org/mailman/listinfo/python-list

Retrieve a GIF's palette entries using Python Imaging Library (PIL)

[Stuart]

I am using the Python Imaging Library (PIL) to process GIF images. I 
need to be able to retrieve the RGB values for each color palette entry.

I see that the 'Image' class has a 'palette' attribute which returns an 
object of type 'ImagePalette'.  However, the documentation is a bit 
lacking regarding how to maniuplate the ImagePalette class to retrieve 
the palette entries' RGB values.  Can anyone point me to a short example?

Thanks!
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Encoding sniffer?

[Stuart Bishop]

[EMAIL PROTECTED] wrote:
> Andreas> Does anyone know of a Python module that is able to sniff the
> Andreas> encoding of text?
> 
> I have such a beast.  Search here:
> 
> http://orca.mojam.com/~skip/python/
> 
> for "decode".
> 
> Skip

We have similar code. It looks functionally the same except that we also:

Check if the string starts with a BOM.
Detects probable ISO-8859-15 using a set of characters common
is ISO-8859-15 but uncommon in ISO-8859-1
Doctests :-)

# Detect BOM
_boms = [
(codecs.BOM_UTF16_BE, 'utf_16_be'),
(codecs.BOM_UTF16_LE, 'utf_16_le'),
(codecs.BOM_UTF32_BE, 'utf_32_be'),
(codecs.BOM_UTF32_LE, 'utf_32_le'),
]

try:
for bom, encoding in _boms:
if s.startswith(bom):
return unicode(s[len(bom):], encoding)
except UnicodeDecodeError:
pass

[...]

# If we have characters in this range, it is probably ISO-8859-15
if re.search(r"[\xa4\xa6\xa8\xb4\xb8\xbc-\xbe]", s) is not None:
try:
return unicode(s, 'ISO-8859-15')
except UnicodeDecodeError:
pass

Feel free to update your available code. Otherwise, I can probably post ours
somewhere if necessary.

-- 
Stuart Bishop <[EMAIL PROTECTED]>
http://www.stuartbishop.net/


signature.asc
Description: OpenPGP digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: How login & download file from remote web site? Passwords a problem?

[Stuart Turner]

[EMAIL PROTECTED] wrote:

> How make a Python script
> 
> 1. login
> 2. type password &
> 3. download file
> 
> all from a **remote web site**?
> 
> I'm not comfortable with *MY* software handling the password part.
> 
> It just seems like trouble if users must tell *my* software their
> password.
> 
> Is there a wise way to solve this or avoid this can of worms?
> 
> Chris

Chris,

As a suggestion, why not look at the urllib and getpass modules.

If you get stuck, mail me and I'll send you over some sample code,

- Stuart

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Is 'everything' a refrence or isn't it?

[Stuart D. Gathman]

On Wed, 04 Jan 2006 10:54:17 -0800, KraftDiner wrote:

> I was under the assumption that everything in python was a refrence...
> 
> so if I code this:
> lst = [1,2,3]
> for i in lst:
>if i==2:
>   i = 4
> print lst
> 
> I though the contents of lst would be modified.. (After reading that
> 'everything' is a refrence.)
> ...
> Have I misunderstood something?

It might help to do a translation to equivalent C:

int _i1 = 1;
int _i2 = 2;
int _i3 = 3;
int _i4 = 4;
int* lst[NLST] = { &_i1,&_i2,&_i3 };
int _idx;   /* internal iterator */
for (_idx = 0; _idx < NLST; ++_idx) {
  int *i = lst[_idx];
  if (*i == *_i2)
i = &_i4;
}
for (_idx = 0; _idx < NLST; ++_idx)
  printf("%d\n",*lst[_idx]);

-- 
  Stuart D. Gathman <[EMAIL PROTECTED]>
Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flamis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: inline function call

[Stuart D. Gathman]

On Wed, 04 Jan 2006 13:18:32 +0100, Riko Wichmann wrote:

> I'm googeling since some time, but can't find an answer - maybe because 
> the answer is 'No!'.
> 
> Can I call a function in python inline, so that the python byte compiler 
> does actually call the function, but sort of inserts it where the inline 
> call is made? Therefore avoiding the function all overhead.

In standard python, the answer is no.  The reason is that all python
functions are effectively "virtual", and you don't know *which* version to
inline.

HOWEVER, there is a slick product called Psyco:

http://psyco.sourceforge.net/

which gets around this by creating multiple versions of functions which
contain inlined (or compiled) code.  For instance, if foo(a,b) is often
called with a and b of int type, then a special version of foo is compiled
that is equivalent performance wise to foo(int a,int b).  Dynamically
finding the correct version of foo at runtime is no slower than normal
dynamic calls, so the result is a very fast foo function.  The only
tradeoff is that every specialized version of foo eats memory.  Psyco
provides controls allowing you to specialize only those functions that
need it after profiling your application.

-- 
  Stuart D. Gathman <[EMAIL PROTECTED]>
Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flamis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Spiritual Programming (OT, but Python-inspired)

[Stuart D. Gathman]

On Mon, 02 Jan 2006 19:05:04 -0500, Steven D'Aprano wrote:


> I don't dare ask where your evidence for this hypothesis is, but I will
> ask what are your reasons for imagining this? What is the chain of
> thought that leads from:
> 
> Step 1: We live in a temporal world.
> 
> to:
> 
> Step N: Our ghost/soul must therefore live in a timeless state.

I can throw in some historical evidence against (assuming you accept the
Gospels as historical, that is - at least they are documents).  Christian
doctrine paraphrased in the programming mindset is that this temporal
world will be rebooted - destroyed and replaced with a "new heavens and
new earth".  The new earth will have time, but is purged of all evil.  The
goal of Christian practice is to cooperate with God as He cleans the
wickedness out of our souls so that we can inhabit the new creation.  The
cleaning experience is not always pleasant.  Taking a hard objective look
at the "goodness" of your behaviour can be humbling and embarrassing.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: sorting with expensive compares?

[Stuart D. Gathman]

On Sat, 24 Dec 2005 15:47:17 +1100, Steven D'Aprano wrote:

> On Fri, 23 Dec 2005 17:10:22 +, Dan Stromberg wrote:
>
>> I'm treating each file as a potentially very large string, and "sorting
>> the strings".
> 
> Which is a very strange thing to do, but I'll assume you have a good
> reason for doing so.

I believe what the original poster wants to do is eliminate duplicate
content from a collection of ogg/whatever files with different names. 
E.g., he has a python script that goes out and collects all the free music
it can find on the web.  The same song may appear on many sites under
different names, and he wants only one copy of a given song.

In any case, as others have pointed out, sorting by MD5 is sufficient
except in cases far less probable than hardware failure - and deliberate
collisions.  E.g., the RIAA creates collision pairs of MP3 files where one
member carries a freely redistributable license, and the other a "copy
this and we'll sue your ass off" license in an effort to trap the unwary.

-- 
  Stuart D. Gathman <[EMAIL PROTECTED]>
Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flamis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: bizarre id() results

[Stuart McGraw]

"Delaney, Timothy (Tim)" <[EMAIL PROTECTED]> wrote in message news:[EMAIL 
PROTECTED]
and several other people responded with similar information.

[...]
Bound methods are created on the fly. 
[...]

Ahhh!  That was the piece I was missing.  Thank you all!
-- 
http://mail.python.org/mailman/listinfo/python-list

bizarre id() results

[Stuart McGraw]

The following was cut and pasted exactly (except for the 
# lines which I added after the fact) from an interactive python 
session in a Window 2000 cmd.exe window. 
 
Can somebody please explain to me what the heck is 
going on?!?!  

Python 2.4.2 (#67, Sep 28 2005, 12:41:11) [MSC v.1310 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> class A:
...  def m1(self): print "m1"
...  def m2(self): print "m2"
...
>>> a = A()
>>> a.m1()
m1
>>> a.m2()
m2
# ok, both methods work and give the expected results
# so i presume they are different methods.
>>> id(a.m1)
9202984
>>> id(a.m2)
9202984
>>> id(a.m1)==id(a.m2)
True
# Huh? They seem to be the same. 
>>> a.m1 is a.m2
False
# But not the same...
>>> a.m1
>
>>> a.m2
>
# Let's look at them in hex...
>>> hex(id(a.m1))
'0x8c6d28'
>>> hex(id(a.m2))
'0x8e7b48'
# Now they are different.  0x8c6d28->9202984, 0x8e7b48->9337672
>>> id(a.m1)
9337672
>>> id(a.m2)
9337672
# Now they are both equal to the second one.
>>> hex(id(a.m1))
'0x8e7b48'
>>> hex(id(a.m2))
'0x8e7b48'
# in hex too.
>>> id

>>> hex

# just double checking!

Why???  This is so bizarre I'm sure I am doing something
really stupid.


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Why are there no ordered dictionaries?

[Stuart McGraw]

"Fredrik Lundh" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED]
> Stuart McGraw wrote
> 
> > > What would improve the Cheese Shop's interface for you?
> >
> > Getting rid of those damn top level links to old versions.
> > Seeing a long list of old versions, when 99% of visitors are
> > only interested in the current version, is just visual noise,
> > and really lame.  Move the old version links onto the page
> > describing the software.
> 
> hmm?  the pypi package automatically hides old versions when
> you post new ones, and it's been that way for ages...
> 
> (which is bloody annoying if you're a package developers, since it
> means that alphas for the next release hides the most recent stable
> version)
> 
> looking at the full index, ZODB seems to be the only package that's
> available in more than just one stable and one development version...

 http://cheeseshop.python.org/pypi?:action=browse&asdf=405
- ClientForm-0.1.17
- ClientForm-0.2.1b
...
- EmPy_3.1
- EmPy_3.1.1
- EmPy_3.2
- EmPy_3.3
...
- FauxIdent-1.1
- FauxIdent-1.2
- FauxIdent-1.2.1
...
Well, it is better than I remember it being a while (year?) ago, my
recollection is that many packages had many, many old versions
listed but now I usualy see only a couple versions.

Hmm, so two versions means one is a development version,
and the other is a stable version?  I did not know that, and did
not see it documented on the site.  I would say documenting 
that would be an interface improvement.

I still think it would be better to have just a package name 
(with current version) listed in the index page(s), and have alternate
versions (old, alpha testing, etc) listed on the package's description
page.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Why are there no ordered dictionaries?

[Stuart McGraw]

"A.M. Kuchling" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED]
[...]
> What would improve the Cheese Shop's interface for you?

Getting rid of those damn top level links to old versions.  
Seeing a long list of old versions, when 99% of visitors are
only interested in the current version, is just visual noise, 
and really lame.  Move the old version links onto the page
describing the software.


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python Book

[Stuart McGraw]

"Stuart McGraw" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED]
> David Beasley's Essential Python (New Riders).  It's a little dated
> now (covers only up to version 2.2) [...]

Oops, that should be "Beazley", "Python Essential Reference", and 
version 2.1.  
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python Book

[Stuart McGraw]

David Beasley's Essential Python (New Riders).  It's a little dated
now (covers only up to version 2.2) but lucid, consise, well organized.
It restricts itself to Python's syntax and semantics and does not waste
time explaining basic programming concepts.

I made several attempts to learn Python but found the Python docs
pretty poor, and the tutorial books I looked at were incredibly ponderous 
and slow.  It wasn't until I got Beasley's book that I could actual find 
info effectively enough to start actually writing Python code.  I still most
often refer to it in preference to the Python docs.

"David Rasmussen" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED]
> What is the best book for Python newbies (seasoned programmer in other 
> languages)?
> 
> /David
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Getting Python Accepted in my Organisation

[Stuart Turner]

Thanks to Everyone for replying - it has given me much food for thought.

- Stuart


Stuart Turner wrote:

> Hi Everyone,
> 
> I'm working hard trying to get Python 'accepted' in the organisation I
> work
> for.  I'm making some good in-roads.  One chap sent me the text below on
> his views of Python.  I wondered if anyone from the group could give me
> some advice on how to respond / if they had been in a similar position.
> 
> Any help appreciated,
> 
> Thanks in advance,
> 
> - Stuart
> 
> 
>  "Python is a scripting language like Perl, awk, tcl, Java etc...  it is
> not quite a fully developed OO language, but does support some OO that
> Perl doesn't.  To be clear, these scripting languages have their place in
> our environment, but they are not full replacements for C#, Java, C,
> etc... because they do not come with the full range of libraries e.g GDI
> libraries.  Python has to be compared to Perl, Awk in order to evaluate
> it. Perl, until recently, did not support threading.  Why would it? it is
> a scripting language and can run async shell commands.  I would be
> interested to learn if Python supports a robust threading model (not just
> a pointer reference to an object), as this is a significant drawback when
> using a scripting language.  CGI only works because the container can
> thread with Perl.  Python is object orientated, but I do not know what
> implementation? Essentially any language with a pointer can claim to be
> OO, although Python does market itself on OO capabilities.  Do you know
> what implementation they have used?
> 
> Lets discuss, as I am not a great fan of Perl and if Python is more
> structured then it is possibly worth promoting."

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Getting Python Accepted in my Organisation

[Stuart Turner]

I'm already using it for a ton of things - I want to try and get broader
acceptance in the organisation for it to be made and 'officially supported
product'.


Stefan Arentz wrote:

> Stuart Turner <[EMAIL PROTECTED]> writes:
> 
>> Hi Everyone,
>> 
>> I'm working hard trying to get Python 'accepted' in the organisation I
>> work
>> for.  I'm making some good in-roads.  One chap sent me the text below on
>> his views of Python.  I wondered if anyone from the group could give me
>> some advice on how to respond / if they had been in a similar position.
> 
> What do you want to use Python for? What do you use now?
> 
>  S.

-- 
http://mail.python.org/mailman/listinfo/python-list

Getting Python Accepted in my Organisation

[Stuart Turner]

Hi Everyone,

I'm working hard trying to get Python 'accepted' in the organisation I work
for.  I'm making some good in-roads.  One chap sent me the text below on
his views of Python.  I wondered if anyone from the group could give me
some advice on how to respond / if they had been in a similar position.

Any help appreciated,

Thanks in advance,

- Stuart


 "Python is a scripting language like Perl, awk, tcl, Java etc...  it is
not quite a fully developed OO language, but does support some OO that Perl
doesn't.  To be clear, these scripting languages have their place in our
environment, but they are not full replacements for C#, Java, C, etc... 
because they do not come with the full range of libraries e.g GDI
libraries.  Python has to be compared to Perl, Awk in order to evaluate it. 
Perl, until recently, did not support threading.  Why would it? it is a
scripting language and can run async shell commands.  I would be interested
to learn if Python supports a robust threading model (not just a pointer
reference to an object), as this is a significant drawback when using a
scripting language.  CGI only works because the container can thread with
Perl.  Python is object orientated, but I do not know what implementation? 
Essentially any language with a pointer can claim to be OO, although Python
does market itself on OO capabilities.  Do you know what implementation
they have used?
  
    Lets discuss, as I am not a great fan of Perl and if Python is more
structured then it is possibly worth promoting."
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: wxpython - passing arg to wx.app OnInit

[Stuart McGraw]

"Peter Hansen" <[EMAIL PROTECTED]> wrote in message news:[EMAIL PROTECTED]
> Stuart McGraw wrote:
> > I have a wxPython app, conventionally structured
> > with a Application class derived from wx.App.
> > My problem is that the app accepts a command
> > line argument that must be acted upon within the
> > OnInit() method of the Application class.  How do
> > I pass it cleanly from main() into app.OnInit()?  In 
> > the simplified example below, dbfn is the piece of 
> > info that is in main() that OnInit() needs to use.
> > Is a global variable is the only way?  :-(
> 
> There are various ways, but the simplest is to accept that sys.argv is
> *already* a "global" and just to access it directly from the
> Application's OnInit() method.
> 
> This wiki page demonstrates:
> http://wiki.wxpython.org/index.cgi/UsingCommandLineArguments
> 
> -Peter

I simplied the my code for posting.  In my real program, the
thing being passed is not a command line argument per se,
but the result of signifigant processing dependent on the 
command line argument.  I do not want to repeat that 
processing in the wx.App method.
Would you elaborate on the other ways?
-- 
http://mail.python.org/mailman/listinfo/python-list

wxpython - passing arg to wx.app OnInit

[Stuart McGraw]

I have a wxPython app, conventionally structured
with a Application class derived from wx.App.
My problem is that the app accepts a command
line argument that must be acted upon within the
OnInit() method of the Application class.  How do
I pass it cleanly from main() into app.OnInit()?  In 
the simplified example below, dbfn is the piece of 
info that is in main() that OnInit() needs to use.
Is a global variable is the only way?  :-(

class Application (wx.App):
def OnInit (self):
oper = Operations (dbfn);
frame = Frame (None, -1, "Title", oper)
self.SetTopWindow (frame)
return True

def main ():
dbfn = sys.args[1]
app = Application (redirect=0)
app.MainLoop ()

Apologies if the answer is (or should be) obvious... this
is my first time using wxWindows and I am still rather new 
to Python.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Postgres PL/Python

[Stuart Bishop]

Ksenia Marasanova wrote:

> I wonder if anyone on this list is using  Python as Postgres
> procedural language. I can't find a place for it i my mind. How would
> a typical MVC web application benefit from it  (besides performance)?
> I understand from the docs that Postgres 7.4 has PL/PythonU -
> unlimited functionality. It sounds like I have the whole Python
> available in Postgres. That means big parts of application logic can
> be moved to stored procedures, and dummy SQL layer becomes something
> else... sounds scary. Any opinions on this?

To start with, PL/PythonU procedures share the same use cases as any other
server side procedure:
- Implementing triggers
- Implementing non-trivial database constraints
- Cases where code will be much faster when run in the RDBMS
  environment rather than in the client. (as an obvious example,
  you are better off running 'SELECT sum(foo) FROM bar' than
  sucking all the results to your client and summing them there.
- Cases where the code can't be rolled out onto the clients. eg.
  it contains secrets, is proprietary and the client runs in
  untrusted environment.
- You need to access a resource only available on the database
  server, such as reading a secret from a file, logging
  information to a central log, making use of cached information
  (if each of your clients implements their own caching, it can
  be inconsistant between them)
- Code you want to run on transaction commit without having to
  put all that transactional logic in your client.
- Providing an interface to data that should not be available
  to clients. eg. validating a password is correct without
  needing to send the plaintext, encrypted or hashed password
  to the client.
- Wierd environments where it is easier to change code in the
  central database than on the clients (eg. the clients are
  installed on laptops that are not always on site or network
  connected, or internal beurocracy makes updating clients
  non trivial).
- Allows logic to be coded and maintained in one place
  and in one language by someone who understands the database
  and the data, rather than having it reimplemented for each
  different client environment, probably incorrectly and
  possibly dangerously

Of course, only some or none of these may be appropriate to your
environment. Or you might have other use cases. I doubt anyone could
actually agree on what 'a typical MVC web application' is. If you have a
trivial system where the database and a single web app run on the same
server and are maintained and developed by the same people, I doubt there is
any reason to use stored procedures. But when you go beyond that you might
find you do.

PL/PythonU procedures have the extra nice use case of being able to share
code between database and clients when the clients are written in Python
(and similarly PL/Perl, PL/Tcl or PL/Java when the clients share the
language). This means your database constraints can use the same algorithms
that the clients are supposed to be using, providing a safety net in case
someone forgets in their code to validate the data before tossing it into
the database or a client for some reason ends up running an old version of
the validatation code.

In general it is less work to keep code client side because it is easier to
modify, test and debug. testing can be done once you have created suitable
fixtures (not trivial, but doable). Debugging is much easier with PL/Python
than, for examplle, PL/SQL because if you structure things correctly you can
run much of your code outside of the database. However, for debugging code
that talks to the internal database APIs you are still stuck with just the
equivalent of 'print' as your toolkit.

-- 
Stuart Bishop <[EMAIL PROTECTED]>
http://www.stuartbishop.net/


signature.asc
Description: OpenPGP digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: how to write thread-safe module ? and pytz

[Stuart Bishop]

nicolas_riesch wrote:
> Does someone know if the module pytz
> (http://sourceforge.net/projects/pytz/) is thread-safe ?
> I have not seen it explicitely stated, and just wanted to be sure, as I
> want to use it.

pytz is thread safe.

> That's because in the file pytz/tzinfo.py, I see global variables
> _timedelta_cache, _datetime_cache, _ttinfo_cache, which are
> dictionnaries and are used as cache.

> I always thought that you must protect shared data with locks when
> multithreading, but I don't see any lock anywhere in pytz.
> However, pytz seems to work well with multiple threads creating various
> timezone objects at the same time.
> I don't understand where the trick is, that allows multiple threads to
> access this module without any locking and that all this seems to work
> without any problem...

Thanks to the global interpreter lock, with the Python builtin types you
only need to maintain a lock if there is a race condition, or if you care
about the race condition. For example, the following is thread safe code:

>>> from threading import Thread
>>> import time
>>> stack = []
>>> stack2 = []
>>> def doit(i):
... stack.append(i)
... time.sleep(0.1)
... stack2.append(stack.pop())
...
>>> threads = [Thread(target=doit, args=(i,)) for i in range(0,100)]
>>> for t in threads: t.start()
...
>>> for t in threads: t.join()
...
>>> len(stack2)
100
>>> stack2
[99, 95, 98, 94, 93, 97, 92, 91, 96, 88, 87, 86, 85, 84, 83, 90, 79, 78, 77,
76, 74, 73, 72, 71, 70, 75, 82, 81, 80, 89, 69, 67, 66, 65, 64, 68, 60, 59,
58, 57, 56, 55, 63, 62, 61, 49, 54, 53, 52, 51, 46, 45, 44, 50, 48, 47, 29,
28, 35, 34, 33, 43, 42, 41, 40, 39, 38, 32, 37, 31, 30, 36, 27, 26, 25, 24,
23, 22, 21, 20, 19, 18, 17, 12, 16, 15, 14, 13, 11, 10, 9, 8, 7, 6, 4, 3, 2,
1, 0, 5]

Note that the value being appended to 'stack2' might not be the value that
was appended to 'stack' in any particular thread - in this case, we don't
care (but is the sort of thing you might need to watch out for).

In the code you mention in pytz, there *is* a race condition. However, if
this condition occurs the side effects are so trivial as to not worry about
locking. ie. if a number of threads call memorized_timedelta(seconds=60)
simultaneously, there is a slight chance that each thread will get a
different timedelta instance. This is extremely unlikely, and the rest of
the code doesn't care at all. If pytz compared the timedeltas using 'is'
instead of '==' at any point, it would be a bug (but it doesn't, so it isn't).

So you can write thread safe Python code without locks provided you are
using the builtin types, and keep a close eye out for race conditions. This
might sound error prone, but it is quite doable provided the critical areas
that are accessing shared objects are kept isolated, short and simple.

Here is an thread unsafe example. Here the mistake is made that the length
of stack will not change after checking it. Also because we don't use the
atomic stack.pop(), two threads might add the same value to stack2:

>>> from threading import Thread
>>> import time
>>> stack = range(0, 50)
>>> stack2 = []
>>> def doit():
... if len(stack) > 0:
... stack2.append(stack[-1])
... time.sleep(0.1)
... del stack[-1]
...
>>> threads = [Thread(target=doit) for i in range(0, 100)]
>>> for t in threads: t.start()
...
Exception in thread Thread-249:
Traceback (most recent call last):
  File "/usr/lib/python2.4/threading.py", line 442, in __bootstrap
self.run()
  File "/usr/lib/python2.4/threading.py", line 422, in run
self.__target(*self.__args, **self.__kwargs)
  File "", line 5, in doit
IndexError: list assignment index out of range



-- 
Stuart Bishop <[EMAIL PROTECTED]>
http://www.stuartbishop.net/


signature.asc
Description: OpenPGP digital signature
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python-Excel: How to paste selected cells (range) to different location on the same sheet in Excel

[Stuart Corrie]

Did you use the excel macro recorder and look at the results to help you 
write that program? It seems to bear all the hallmarks of code generated 
that way. The macro recorder can be great, but almost always it is 
possible to speed up code by altering it afterwards, to condense and 
speed up the VBA code. A slightly tighter version of your code is shown 
below:

import win32com.client
xl=win32com.client.Dispatch("Excel.Application")
xl.Visible=1
wb = xl.Workbooks.Add( )
sh=wb.Worksheets(1)
sh.Cells(1,1).Value = "Hello World!"
sh.Cells(3,3).Value = "Hello World!"
sh.Range(sh.Cells(1,1),sh.Cells(3,3)).Copy(sh.Cells(4,1))
sh.Range(sh.Cells(4,1),sh.Cells(6,3)).Select()

The first 7 lines are identical to yours. The way you work with excel in 
when entering data and formulae interactively, and so the way the macro 
recorder must work, involves lots of selection (or activation for 
switching workbooks) which can be slow and often not required when using 
code to manipulate excel's objects through COM from an external language 
or VBA in excel. Line 7 in your program 
"sh.Range(sh.Cells(1,1),sh.Cell­s(3,3)).Select()" actually does nothing 
because the range is specified again in line 8.

The copy method of a range object can take a parameter for a paste 
destination. The macro recorder will never generate code like this 
because it must record the selection between the copy and the paste 
operations.

The only minor drawback of the revised line 7 is that the range selected 
after the paste operation is the parameter given to copy not the data 
pasted, if this is required, line 8 fixes this.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Ideas for Python project?

[Stuart Turner]

Jon Hewer wrote:

> Hi
> 
> I'm pretty new to Python, and recently been working my way through
> Dive Into Python, and I'm currently writing a really simple rss reader
> purely to get familiarised with the language.  I want to move onto
> something a little more challenging, but I'm stuck for ideas on what
> to do.
> 
> I'm after a project that can start quite simple, but has lots of room
> for expansion when I get more confident with Python.  Ideally
> something which would start as a command line program, and then when I
> feel adventurous I could build a GUI for it.
> 
> I have just bought the Foundations of Python Network Programming book,
> so maybe something network related would be good?
> 
> Cheers
> Jon

I saw a competition in Linux format - may be something to consider.

- Stuart
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Friend wants to learn python

[Stuart Turner]

EnderLocke wrote:

> I have a friend who wants to learn python programming. I learned off
> the internet and have never used a book to learn it. What books do you
> recommend?
> 
> Any suggestions would be appreciated.


I've just been learning Python and used a combination of 'Python in a
Nutshell, Python Cookbook and whatever I could find on the web'

I also find giving yourself a *real* programming task makes you learn a lot
quicker!

- Stuart

-- 
http://mail.python.org/mailman/listinfo/python-list

Tix example code?

[Stuart McGraw]

I am having trouble trying to figure out how to use Tix widgets
in a Python program.  The documentation is scanty and in 
TCL syntax with no examples.  Specifically, I'm trying to use 
a Tix ComboBox widget and haven't been able to figure out 
how to initialize the set of values in the widget's list. 

Does anyone have some (or a pointer to) simple example 
python code that uses TIX widgets (especially the ComboBox)
including subwidgets?  Thanks!...

(And, yes, I looked at the "Mapping Basic Tk into Tkinter "
rosetta stone web page but it did not provide much enlightenment.)
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: DB API 2.0 and transactions

[Stuart Bishop]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1

Magnus Lycka wrote:

> Another option would be to investigate if any of the other postgreSQL
> drivers have a more correct behaviour. The non-standard behaviour that
> you describe it obvious from the pgdb source. See:
> http://www.pygresql.org/cvsweb.cgi/pygresql/module/pgdb.py?rev=1.27

fwiw psycopg 1.1.18 has the correct behavior (as usual).

I can add this test to the DB-API 2.0 Anal Compliance Test Suite if we want,
although it sounds like it is only an issue with PostgreSQL drivers.

- --
Stuart Bishop <[EMAIL PROTECTED]>
http://www.stuartbishop.net/
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)

iD8DBQFCsk3oAfqZj7rGN0oRAjHAAJ4kQzxJXFW6hX6Q1t896fMzT0EUjACgkBhw
X4wB17+4FwO9TsKpiIBJB50=
=mYfn
-END PGP SIGNATURE-
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Memory Usage

[Stuart McGarrity]

Do you have a page file?

The Mem column should be RAM usage and not total working set. Some of it 
could be swapped to the page file. A free tool like process explorer can 
give you better informaton than the task manager.

"rbt" <[EMAIL PROTECTED]> wrote in message 
news:[EMAIL PROTECTED]
> Would a Python process consume more memory on a PC with lots of memory?
>
> For example, say I have the same Python script running on two WinXP 
> computers that both have Python 2.4.0. One computer has 256 MB of Ram 
> while the other has 2 GB of Ram. On the machine with less Ram, the process 
> takes about 1 MB of Ram. On the machine with more Ram, it uses 9 MB of 
> Ram.
>
> Is this normal and expected behavior?
>
> Thanks,
>
> rbt 


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Tabnanny really useful?

[Stuart Bishop]

-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Franz Steinhaeusler wrote:
| Not really soo useful, because most syntax and also indentation errors
| are actually detected by invoking python, i.e. the command compile.
| But as combination for this: yes why not.
| I looked for Stanis spe editor, which uses a combination of these two.
Just because Python compiles and runs something does not mean that there
are no indentation errors. tabnanny works great because it removes all
ambiguous cases so you never have to worry about it. We use it as part
of our test suite to guard against misconfigured editors.
(again using '.' instead of space) Why would you want to allow '.\tprint
foo' in Python source when you could clean it up to be the totally
unambigous 'print foo'. As a more rational example, would you
want to allow '\tprint foo', which will totally confuse someone with
their editor misconfigured with 4 char tabs?
- --
Stuart Bishop <[EMAIL PROTECTED]>
http://www.stuartbishop.net/
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.2.5 (GNU/Linux)
iD8DBQFByp4nAfqZj7rGN0oRAoq9AJ4scAe0A3aFnx7jRZuqtRXXaxAcRgCfYkVf
FbH2HBFA4/44KgZfpiPuvNU=
=nyQt
-END PGP SIGNATURE-
--
http://mail.python.org/mailman/listinfo/python-list

Re: Help beautify ugly heuristic code

[Stuart D. Gathman]

On Fri, 10 Dec 2004 22:03:20 +, JanC wrote:

> Stuart D. Gathman schreef:
> 
>> I have a function that recognizes PTR records for dynamic IPs.  There

> Did you also think about ISPs that use such a PTR record for both dynamic 
> and fixed IPs?

There seems to be a lot of misunderstanding about this.  I am not blocking
anyones mail because they have a dynamic looking PTR.  I simply don't
accept such a PTR as MTA authentication.  You see, MTAs *SHOULD* provide a
fully qualified domain as their HELO name which resolves to the IP of the
MTA.  Sadly, however, many internet facing MTAs don't do this, but I
accept a meaningful PTR as a substitute.  I also waive the requirement for
MTA authentication if the MAIL FROM has an SPF record
(http://spf.pobox.com).

So, if your MTA complies with RFC HELO recommendations, you'll have no
trouble sending me mail. You can even use a dynamic IP with a dynamic DNS
service.

I 'do* block PTR names of "." or "localhost".  I would like to block all
single word HELO names - but there are too many clueless mail admins out
there.  People seem to be unsure of what to send for HELO.

-- 
  Stuart D. Gathman <[EMAIL PROTECTED]>
Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flamis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Help beautify ugly heuristic code

[Stuart D. Gathman]

On Thu, 09 Dec 2004 00:01:36 -0800, Lonnie Princehouse wrote:

> I believe you can still do this with only compiling a regex once and
> then performing a few substitutions on the hostname.

Cool idea.  Convert ip matches to fixed patterns before matching a fixed
regex. The leftovers like shaw cable (which has the MAC address of the
cable modem instead of the IP) can still be handled with regex patterns.

I had an idea last night to compile 254 regexes, one for each possible
last IP byte - but I think your idea is better.

Mitja suggested a socring system reminiscent of SpamAssassin.

That gives me a few things to try.

-- 
  Stuart D. Gathman <[EMAIL PROTECTED]>
Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flamis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Creating Fixed Length Records

[Stuart D. Gathman]

On Wed, 08 Dec 2004 17:29:19 -0600, Greg Lindstrom wrote:


> One thought I had, which might lead to an addition to the language, was 
> to use the struct module.  If I could feed the pack method a format 
> string then a tuple of values (instead of individual values), then I 
> could create the format string once, then pass it a tuple with the 
> values for that record.  Just a thought. 

The language extension has been around since the early days. It was 
originally called 'apply'.  But now is called '*'.

>>> import struct
>>> fmt = '8s6s2s'
>>> v = ('foo','bar','AA')
>>> struct.pack(fmt,*v)
'foo\x00\x00\x00\x00\x00bar\x00\x00\x00AA'

-- 
  Stuart D. Gathman <[EMAIL PROTECTED]>
Business Management Systems Inc.  Phone: 703 591-0911 Fax: 703 591-6154
"Confutatis maledictis, flamis acribus addictis" - background song for
a Microsoft sponsored "Where do you want to go from here?" commercial.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Help beautify ugly heuristic code

[Stuart D. Gathman]

On Wed, 08 Dec 2004 19:52:53 -0500, Lonnie Princehouse wrote:

> I don't think a Bayesian classifier is going to be very helpful here,
> unless you have tens of thousands of examples to feed it, or unless it

We do have tens of thousands of examples to feed it.

> The series of if host.find(...) lines in is_dynip() is equivalent to a
> regular expression, but much more expensive to execute because of all

It is not equivalent, because the patterns are based on the IP address.
As I mentioned before, I tried building a custom regex from the IP for
each test - but compiling the regex is way too slow to be done for each
test.

> For IP addresses, you really just need a mechanism to filter blocks of
> IP addresses.  It might be easiest to first convert them into hex and
> then make liberal use of [0-f] in regular expressions.

The point of the ip address is *not* to recognize ip addresses.  The
point is to look for transformations of the ip address in the hostname.
This gives a *huge* bang for the buck.  I have been working on this
problem for a while.  If the hostname has a transformation of the ip
address - it is (almost certainly) a dynamic address.  The ISPs are very
creative in their transformations, using the parts of the ip in various
orders and encoding in hex, base64, decimal with or without zerofill, and
even roman numerals.

The regex engine is just not powerful enough to handle parameterized
regexe (that I know of).
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Help beautify ugly heuristic code

[Stuart D. Gathman]

On Wed, 08 Dec 2004 18:39:15 -0500, Lonnie Princehouse wrote:

> Regular expressions.
> 
> It takes a while to craft the expressions, but this will be more
> elegant, more extensible, and considerably faster to compute (matching
> compiled re's is fast).

I'm already doing that with the rehmac regex.  I like your idea for making
it more readable, though.  Looking for permutations of the IP address
gives much more bang for the line of code than most host only regexes
since it is ISP independent.  At least one ISP uses roman numerals to code
the IP for their dynamic addresses!  I tried matching a custom regex
computed from the IP, but compiling the regex for each test was too slow.

I could keep adding more patterns, but I was hoping for a tool that
"learns" from a database of preclassified examples how to recognize the
pattern.  And the resulting data would be reasonably compact.  I don't ask
for much, do I?  A Bayesian classifier would have too big of a database, I
think.  I've seen neural nets do amazing things with only 100 or so
neurons - a small weight database. But they are slow in software.

I have posted 10K preclassified (by current algorithm) examples here:
http://bmsi.com/python/dynip.samp
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Help beautify ugly heuristic code

[Stuart D. Gathman]

On Wed, 08 Dec 2004 18:00:06 -0500, Mitja wrote:

> On Wed, 08 Dec 2004 16:09:43 -0500, Stuart D. Gathman <[EMAIL PROTECTED]>
> wrote:
> 
>> I have a function that recognizes PTR records for dynamic IPs Here
>> is the very ugly code so far.
>> ...
>> # examples we don't yet recognize:
>> ...
> 
> This doesn't help much; post example of all the possible patterns you
> have to match (kind of like the docstring at the beginning, only more
> elaborate), otherwise it's hard to know what kind of code you're trying
> to implement.

This is a heuristic, so there is no exhaustive list or hard rule.
However, I have posted 23K+ examples at http://bmsi.com/python/dynip.samp
with DYN appended for examples which the current algorithm classifies
as dynamic.

Here are the last 20 (which my subjective judgement says are correct):

65.112.76.15usfshlxmx01.myreg.net
201.128.108.41  dsl-201-128-108-41.prod-infinitum.com.mx DYN
206.221.177.128 mail128.tanthillyingyang.com
68.234.254.147  68-234-254-147.stmnca.adelphia.net DYN
63.110.30.30mx81.goingwiththe-flow.info
62.178.226.189  chello062178226189.14.15.vie.surfer.at DYN
80.179.107.85   80.179.107.85.ispeednet.net DYN
200.204.68.52   200-204-68-52.dsl.telesp.net.br DYN
12.203.156.234  12-203-156-234.client.insightBB.com DYN
200.83.68.217   CM-lconC1-68-217.cm.vtr.net DYN
81.57.115.43pauguste-3-81-57-115-43.fbx.proxad.net DYN
64.151.91.225   sv2a.entertainmentnewsclips.com
64.62.197.31teenfreeway.sparklist.com
201.9.136.235   201009136235.user.veloxzone.com.br DYN
66.63.187.9191.asandox.com
83.69.188.198   st11h07.ptambre.com
66.192.199.217  66-192-199-217.pyramidcollection.org DYN
69.40.166.49h49.166.40.69.ip.alltel.net DYN
203.89.206.62   smtp.immigrationexpert.ca
80.143.79.97p508F4F61.dip0.t-ipconnect.de DYN
-- 
http://mail.python.org/mailman/listinfo/python-list