Re: Bug in string.find; was: Re: Proposed PEP: New style indexing,was Re: Bug in slice type

[Paul Rubin]

"Terry Reedy" <[EMAIL PROTECTED]> writes:
> The fact that the -1 return *has* lead to bugs in actual code is the 
> primary reason Guido has currently decided that find and rfind should go. 
> A careful review of current usages in the standard library revealed at 
> least a couple bugs even there.

Really it's x[-1]'s behavior that should go, not find/rfind.

Will socket.connect_ex also go?  How about dict.get?  Being able to
return some reasonable value for "failure" is a good thing, if failure
is expected.  Exceptions are for unexpected, i.e., exceptional failures.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Writing Multithreaded Client-Server in Python.

[Paul Rubin]

"Sidd" <[EMAIL PROTECTED]> writes:
>I tried finding and example of multithreaded client-serve program in
> python. Can any one please tell me how to write a multithreaded
> client-server programn in python such that
> 1.It can handle multiple connections
> 2.It uses actual threads and not select() or some other function

If you mean multiple threads on the server side, see the SocketServer
module and its ThreadingMixin class.  You may have to look at the
source code since up until recently the docs were incomplete.  There
is a big helpful comment at the top of SocketServer.py which recently
got merged into the library reference manual.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: trictionary?

[[EMAIL PROTECTED]]

Bengt Richter wrote:
> On 28 Aug 2005 18:54:40 -0700, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> wrote:
>
> >Steven Bethard wrote:
> >> Adam Tomjack wrote:
> >> > Steven Bethard wrote:
> >> > ...
> >> >> Using a two element list to store a pair of counts has a bad code
> >> >> smell to me.
> >> > ...
> >> >
> >> > Why is that?
> >>
> >> Note that "code smell"[1] doesn't mean that something is actually wrong,
> >> just that it might be.  In Python, pairs are usually handled with
> >> tuples[2], but tuples would be inconvenient in this case, since the
> >> first value must be modified.  Declaring a class with two attributes as
> >> you suggested is often a good substitute, but if the OP's code is really
> >> what it looks like, I get another code smell because declaring a class
> >> to be used by only 10 lines of code seems like overkill.
> >>
> >> I also get a code smell from a dict holding two-element lists because
> >> I've been writing in Python and answering questions on the Python-list
> >> for a couple of years now, and I've never needed one yet. ;-)
> >>
> >> STeVe
> >>
> >> [1]http://en.wikipedia.org/wiki/Code_smell
> >> [2]http://www.python.org/doc/faq/general.html#why-are-there-separate-tuple-and-list-data-types
> >
> >Could you do me a favor and see what this smells like?
> >
> >I put some data in a csv file (with headers) and this will
> >bring it in quite simply as a dictionary with [names] as
> >keys and an attribute dictionary as the value.
> >
> >py_monsters.csv:
> >
> >name,hardiness,agility,friend,courage,room,weight,special_def_odds,armor,weapon,odds,dice,side,hits,reaction,desc
> >PIRATE,5,20,0,10,26,300,0,0,11,60,1,10,0,"not met",You see a man with a
> >beard and a brass ring in his ear.  He is wearing clothes  made of silk
> >and is wielding a very fancily engraved sword.
> >
> >
> >import csv
> >temp1 = []
> >temp2 = []
> >reader = csv.reader(file(r"py_monsters.csv"))
> >for rec in reader:
> > temp1.append(rec)
> >for i in temp1[1:]:
> > temp2.append((i[0],dict(zip(temp1[0][1:],i[1:]
> >monsters = dict(temp2)
> >
> >This gives me what I want
> >
> >[('PIRATE', {'reaction': 'not met',
> >'agility': '20',
> >'room': '26',
> >'weight': '300',
> >'armor': '0',
> >'weapon': '11',
> >'hits': '0',
> >'side': '10',
> >'special_def_odds': '0',
> >'courage': '10',
> >'hardiness': '5',
> >'desc': 'You see a man with a beard and a brass ring in his ear.
> >He is wearing clothes  made of silk and is wielding a very fancily
> >engraved sword.',
> >'odds': '60',
> >'friend': '0',
> >'dice': '1'})]
> >
> >so that I can now write code like
> >
> >if monsters['PIRATE']['reaction']=='not met':
> >print monsters['PIRATE']['desc']
>
> I think if the field names are legal python names, I would rather write that 
> 'if ...'
> without all that line noise ;-) E.g., using simple classes (and effectively 
> using
> their attribute instance dicts essentially the way you used raw dicts), you 
> could write
>
> if monsters.PIRATE.reaction == 'not met':
> print monsters.PIRATE.desc
>
> Also, if the reaction, agility, etc list is fixed, you could use __slots__ for
> better efficiency. Also, your __init__ method for the latter could convert
> strings to ints for handier use later (unless csv already does that in the 
> mode
> you are using).

No, it doesn't as I found out shortly after posting that. What was I
thinking?
I've got it fixed for the moment, but that idea of classes sounds
interesting.

>
> From the above code I infer that you can do define the requisite classes, so
> I'll leave it to you ;-)

Guess I'll have to actually start reading the manuals.

> Note that attribute access opens the door to having
> default values as class variables, and using properties to retrieve 
> dynamically
> calculated values by name. If you are not familiar with those aspects of 
> classes,
> your progammer.agility value can increase markedly with a little study ;-)
>
> E.g., a name like monsters.random_elf could live right
> alongside PIRATE and return one a randomly selected elf from a an internal 
> list
> of elves or via random selection from a list of elf names defined at the same
> level as PIRATE, etc. etc. You could also dynamically configure these guys
> according to distance to food and time since last meal, and if you are
> carrying food, etc.
>
> With suitable classes you could put instances in various "spaces" defined
> by other classes, that define geometric or social or other interactions.

Well, this is a port of The Wonderful World of Eamon, which isn't quite
that ambitious,
but thanks for the idea about classes. It's one of those many things I
just skim past
without ever understanding. Now would be a good time to start learning
it.

>
> >
> >instead of using stupid index numbers.
> >
> >But the loader seems to have a kind of perl-like odor to it,
> >i.e., next week I won't understand what it does.
> >
> If you have to do something tricky, choose names wisely and
> comment the no

Re: Bug in string.find; was: Re: Proposed PEP: New style indexing,was Re: Bug in slice type

[Terry Reedy]

"Steve Holden" <[EMAIL PROTECTED]> wrote in message 
news:[EMAIL PROTECTED]
> Antoon Pardon wrote:
>> So what happens if you have a module that is collecting string-index
>> pair, colleted from various other parts. In one part you
>> want to select the last letter, so you pythonically choose -1 as
>> index. In an other part you get a result of find and are happy
>> with -1 as an indictation for an invalid index. Then these
>> data meet.
>>
> That's when debugging has to start. Mixing data of such types is
> somewhat inadvisable, don't you agree?
>
> I suppose I can't deny that people do things like that, myself included,
> but mixing data sets where -1 is variously an error flag and a valid
> index is only going to lead to trouble when the combined data is used.

The fact that the -1 return *has* lead to bugs in actual code is the 
primary reason Guido has currently decided that find and rfind should go. 
A careful review of current usages in the standard library revealed at 
least a couple bugs even there.

Terry J. Reedy



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: global interpreter lock

[Bryan Olson]

Mike Meyer wrote:
 > Bryan Olson <[EMAIL PROTECTED]> writes:
 >> > Bryan Olson writes:
 >> > Trivially, an 'if' statement that depends upon input
 >> >>data is statically predictable. Use of async I/O means makes the
 >> >>programs execution dependent upon external timing.
 >>Mike Meyer wrote:
 >> > Yes, but that depenency is tied to a single point - the select
 >> > call. The paths after that are statically predictable. This makes the
 >> > code very managable.
 >>Wow -- I could not disagree more. Returning back to some single
 >>point for every possibly-blocking operation is painful to manage
 >>even for simple GUIs, and humanly intractable for sophisticated
 >>services.
 >
 > I'd be interested in what you're trying to do that winds up as
 > unmanagable.

I'd like to call a utility in someone else's code, but it might
do something that could block. I find re-writing everyone's code
into a state-machine that can trap back to the central I/O loop
and later resume where it left off, to be hard to manage. When
I'm writing my own base classes, I find it hard to support the
back-to-the-I/O-loop-and-resume thing so that method over-rides
can call blocking operations when they need to.


 > There are clearly things select+async IO is unsuitable
 > for. You may be running into problems because you're trying to use it
 > in such an environment. For instance, it's not clear to me that it
 > will work well for any kind of GUI programming, though I've had good
 > look with it for command line interfaces.

Uh, not sure where you're coming from there. Are you unaware of
the 'event loop' in the GUI's, or unaware that it's async I/O.

 >>Select is certainly useful, but it scales badly and isn't as
 >>general as better tools.
 >
 > It can't take advantage of multiple CPUs. I've not run into scaling
 > problems on single-CPU systems.

Select() is linear-time in the number of sockets to be checked
(not just the number found to be ready). There's a good write-up
of the problem and solutions Google-able as "The C10K problem".


 >> > [...] I'm calling the tools available in most programming
 >> > languages for dealing with it primitive.
 >> > We need better tools.
 >>Agreed, but if 'select' is someone's idea of the state of the
 >>art, they have little clue as to the tools already available.
 >
 > Well, share!

Uh, where have you been? I keep explaining that concurrency
systems have improved vastly in recent years. For a long time,
the most sophisticated software services generally have used
multiple lines of execution, and now that's mostly in the form
of threads. No one actually disagrees, but they go right on
knocking the modern methods.


-- 
--Bryan
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: aproximate a number

[Grant Edwards]

On 2005-08-30, Devan L <[EMAIL PROTECTED]> wrote:
> Grant Edwards wrote:
>> On 2005-08-30, Devan L <[EMAIL PROTECTED]> wrote:
>> >
>> > RoundToInt(2.0) will give you 3.
>>
>> That's what the OP said he wanted.  The next bigger integer
>> after 2.0 is 3.
>
> It's not really clear whether he wanted it to round up or to go to the
> next biggest integer because he says he has bad english. I can't think
> of a particular use of returning the next bigger integer.

You're probably right.  I suspect what he really wants is

i = int(math.ceil(x))

-- 
Grant Edwards   grante Yow!  Is it FUN to be
  at   a MIDGET?
   visi.com
-- 
http://mail.python.org/mailman/listinfo/python-list

Writing Multithreaded Client-Server in Python.

[Sidd]

Hi,
   I tried finding and example of multithreaded client-serve program in
python. Can any one please tell me how to write a multithreaded
client-server programn in python such that
1.It can handle multiple connections
2.It uses actual threads and not select() or some other function

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: aproximate a number

[Devan L]

Grant Edwards wrote:
> On 2005-08-30, Devan L <[EMAIL PROTECTED]> wrote:
> >
> > RoundToInt(2.0) will give you 3.
>
> That's what the OP said he wanted.  The next bigger integer
> after 2.0 is 3.
>
> --
> Grant Edwards   grante Yow!  I'd like TRAINED
>   at   SEALS and a CONVERTIBLE on
>visi.commy doorstep by NOON!!

It's not really clear whether he wanted it to round up or to go to the
next biggest integer because he says he has bad english. I can't think
of a particular use of returning the next bigger integer.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: [Jython-users] Re: [Jython-dev] Parsing grammar for jython available (compatible with python 2.4)

[could ildg]

thanks.
pydev is very nice, I like it.
But sometimes the code completion will not work.On 8/30/05, Frank Wierzbicki <[EMAIL PROTECTED]> wrote:

The code, as well as the files for javacc and asdl (both were changed)

are available in the pydev cvs at sourceforge, in theorg.python.pydev.parser module (the packages should be the same jythonuses -- org.python.parser and org.python.parser.ast).The tests I used are available in the tests source folder
(org.python.pydev.parser.PyParserTest)Hey -- that's great!
Unfortunately we probably will not be able to integrate 2.4 features in
until after we get a 2.3 version stable... But I hope you can remind us
of this work (or I can remember it) when we do get to that point. 
If you want to leave a strong reminder you could submit your changes as
a patch -- but I mean this as only the gentlest of nudges since it will
likely be quite a while before we could really look at it.

Thanks,
Frank
 


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: aproximate a number

[Grant Edwards]

On 2005-08-30, Devan L <[EMAIL PROTECTED]> wrote:

>>> Hi all. I'd need to aproximate a given float number into the
>>> next (int) bigger one. Because of my bad english I try to
>>> explain it with some example:
>>>
>>> 5.7 --> 6
>>> 52.987 --> 53
>>> 3.34 --> 4
>>> 2.1 --> 3
>>
>> The standard way to do this is thus:
>>
>> def RoundToInt(x):
>> """ Round the float x to the nearest integer """
>> return int(round(x+0.5))
>>
>> x = 5.7
>> print x, '-->', RoundToInt(x)
>> x = 52.987
>> print x, '-->', RoundToInt(x)
>> x = 3.34
>> print x, '-->', RoundToInt(x)
>> x = 2.1
>> print x, '-->', RoundToInt(x)
>>
>> 5.7 --> 6
>> 52.987 --> 53
>> 3.34 --> 4
>> 2.1 --> 3
>
> RoundToInt(2.0) will give you 3.

That's what the OP said he wanted.  The next bigger integer
after 2.0 is 3.

-- 
Grant Edwards   grante Yow!  I'd like TRAINED
  at   SEALS and a CONVERTIBLE on
   visi.commy doorstep by NOON!!
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: aproximate a number

[Devan L]

Thomas Bartkus wrote:
> On Sun, 28 Aug 2005 23:11:09 +0200, billiejoex wrote:
>
> > Hi all. I'd need to aproximate a given float number into the next (int)
> > bigger one. Because of my bad english I try to explain it with some example:
> >
> > 5.7 --> 6
> > 52.987 --> 53
> > 3.34 --> 4
> > 2.1 --> 3
> >
>
> The standard way to do this is thus:
>
> def RoundToInt(x):
> """ Round the float x to the nearest integer """
> return int(round(x+0.5))
>
> x = 5.7
> print x, '-->', RoundToInt(x)
> x = 52.987
> print x, '-->', RoundToInt(x)
> x = 3.34
> print x, '-->', RoundToInt(x)
> x = 2.1
> print x, '-->', RoundToInt(x)
>
> 5.7 --> 6
> 52.987 --> 53
> 3.34 --> 4
> 2.1 --> 3

RoundToInt(2.0) will give you 3.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Using select on a unix command in lieu of signal

[Jp Calderone]

On 29 Aug 2005 17:57:34 -0700, rh0dium <[EMAIL PROTECTED]> wrote:
>So here's how I solved this..  It's seems crude - but hey it works.
>select not needed..
>
>def runCmd( self, cmd, timeout=None ):
>self.logger.debug("Initializing function %s - %s" %
>(sys._getframe().f_code.co_name,cmd) )
>command = cmd + "\n"
>
>child = popen2.Popen3(command)
>t0 = time.time()
>
>out = None
>while time.time() - t0 < timeout:
>if child.poll() != -1:
>self.logger.debug("Command %s completed succesfully" %
>cmd )
>out = child.poll()
>results = "".join(child.fromchild.readlines())
>results = results.rstrip()
>break
>print "Still waiting..", child.poll(), time.time() -
>t0, t0
>time.sleep(.5)
>
>if out == None:
>self.logger.warning( "Command: %s failed!" % cmd)
>kill = os.kill(child.pid,9)
>self.logger.debug( "Killing command %s - Result: %s" %
>(cmd, kill))
>out = results = None
>
>else:
>
>self.logger.debug("Exit: %s Reullts: %s" % (out,results))
>
>child.tochild.close()
>child.fromchild.close()
>return out,results
>
>Comments..
>

Here's how I'd do it...


from twisted.internet import reactor, protocol, error

class PrematureTermination(Exception):
"""Indicates the process exited abnormally, either by receiving an 
unhandled signal or with a non-zero exit code.
"""

class TimeoutOutputProcessProtocol(protocol.ProcessProtocol):
timeoutCall = None
onCompletion = None

def __init__(self, onCompletion, timeout=None):
# Take a Deferred which we will use to signal completion (successful 
# or otherwise), as well as an optional timeout, which is the maximum 
# number of seconds (may include a fractional part) for which we will 
# await the process' completion.
self.onCompletion = onCompletion
self.timeout = timeout

def connectionMade(self):
# The child process has been created.  Set up a buffer for its output,
# as well as a timer if we were given a timeout.
self.output = []
if self.timeout is not None:
self.timeoutCall = reactor.callLater(
self.timeout, self._terminate)

def outReceived(self, data):
# Record some data from the child process.  This will be called 
# repeatedly, possibly with a large amount of data, so we use a list 
# to accumulate the results to avoid quadratic string-concatenation 
# behavior.  If desired, this method could also extend the timeout: 
# since it is producing output, the child process is clearly not hung;
# for some applications it may make sense to give it some leeway in 
# this case.  If we wanted to do this, we'd add lines to this effect:
# if self.timeoutCall is not None:
# self.timeoutCall.delay(someNumberOfSeconds)
self.output.append(data)

def _terminate(self):
# Callback set up in connectionMade - if we get here, we've run out of 
# time.  Error-back the waiting Deferred with a TimeoutError including
# the output we've received so far (in case the application can still 
# make use of it somehow) and kill the child process (rather 
# forcefully - a nicer implementation might want to start with a 
# gentler signal and set up another timeout to try again with KILL).
self.timeoutCall = None
self.onCompletion.errback(error.TimeoutError(''.join(self.output)))
self.onCompletion = None
self.output = None
self.transport.signalProcess('KILL')

def processEnded(self, reason):
# Callback indicating the child process has exited.  If the timeout 
# has not expired and the process exited normally, callback the 
# waiting Deferred with all our results.  If we did time out, nothing 
# more needs to be done here since the Deferred has already been 
# errored-back.  If we exited abnormally, error-back the Deferred in a
# different way indicating this.
if self.onCompletion is not None:
# We didn't time out
self.timeoutCall.cancel()
self.timeoutCall = None

if reason.check(error.ProcessTerminated):
# The child exited abnormally
self.onCompletion.errback(
PrematureTermination(reason, ''.join(self.output)))
else:
# Success!  Pass on our output.
self.onCompletion.callback(''.join(self.output)))

# Misc. cleanup
self.onCompletion = None
self.output = None

def runCmd(executable, args, timeout=None, **kw):
d = defer.Deferred()
p = TimeoutOutputProcessProtocol(d, timeout)
   

Re: aproximate a number

[Thomas Bartkus]

On Sun, 28 Aug 2005 23:11:09 +0200, billiejoex wrote:

> Hi all. I'd need to aproximate a given float number into the next (int) 
> bigger one. Because of my bad english I try to explain it with some example:
> 
> 5.7 --> 6
> 52.987 --> 53
> 3.34 --> 4
> 2.1 --> 3
> 

The standard way to do this is thus:

def RoundToInt(x):
""" Round the float x to the nearest integer """
return int(round(x+0.5))

x = 5.7
print x, '-->', RoundToInt(x)
x = 52.987
print x, '-->', RoundToInt(x)
x = 3.34
print x, '-->', RoundToInt(x)
x = 2.1
print x, '-->', RoundToInt(x)

5.7 --> 6
52.987 --> 53
3.34 --> 4
2.1 --> 3


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: File parser

[Rune Strand]

It's not clear to me from your posting what possible order the tags may
be inn. Assuming you will always END a section before beginning an new,
eg.

it's always:

A
 some A-section lines.
END A

B
some B-section lines.
END B

etc.

And never:

A
 some A-section lines.
B
some B-section lines.
END B
END A

etc.

is should be fairly simple. And if the file is several GB, your ought
to use a generator in order to overcome the memory problem.

Something like this:


def make_tag_lookup(begin_tags):
  # create a dict with each {begin_tag : end_tag}
  end_tags = [('END ' + begin_tag) for begin_tag in begin_tags]
  return dict(zip(begin_tags, end_tags))


def return_sections(filepath, lookup):
  # Generator returning each section

  inside_section = False

  for line in open(filepath, 'r').readlines():
line = line.strip()
if not inside_section:
  if line in lookup:
inside_section = True
data_section = []
section_end_tag = lookup[line]
section_begin_tag = line
data_section.append(line) # store section start tag
else:
  if line == section_end_tag:
data_section.append(line) # store section end tag
inside_section = False
yield data_section # yield entire section

  else:
data_section.append(line) #store each line within section


# create the generator yielding each section
#
sections = return_sections(datafile,
make_tag_lookup(list_of_begin_tags))

for section in sections:
  for line in section:
print line
  print '\n'

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: File parser

[William Park]

Angelic Devil <[EMAIL PROTECTED]> wrote:
> BAR
> END BAR
> 
> FOOBAR
> END FOOBAR

man csplit  

-- 
William Park <[EMAIL PROTECTED]>, Toronto, Canada
ThinFlash: Linux thin-client on USB key (flash) drive
   http://home.eol.ca/~parkw/thinflash.html
BashDiff: Super Bash shell
  http://freshmeat.net/projects/bashdiff/
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Jargons of Info Tech industry

[John Bokma]

Alan Balmer <[EMAIL PROTECTED]> wrote:

> On 29 Aug 2005 21:12:13 GMT, John Bokma <[EMAIL PROTECTED]> wrote:
> 
>>> Now, go away. And please, stay away.
>>
>>Like I already said, it doesn't work that way.
> 
> Goodbye, John. Filters set.

Saidly you didn't get the message. Moreover you think that the Usenet 
/needs/ a public ploink message. Get a clue. People like you add more noise 
to Usenet compared to a thread which runs a bit wide.

-- 
John   Small Perl scripts: http://johnbokma.com/perl/
   Perl programmer available: http://castleamber.com/
Happy Customers: http://castleamber.com/testimonials.html

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: new line

[Peter Hansen]

Kuljo wrote:
> Kuljo wrote:
>>I'm so sorry to bore you with this trivial problem. Allthou: I have string
>>having 0x0a as new line, but I should have \n instead.

> I have found this in the meantime:
nl="\\"+"n"

Note: this is unnecessary.  You could just do nl='\\n' instead, and you 
don't need the variable "nl" either, which by the way is confusingly 
named.  Your variable "nl" is actually bound to the two character 
sequence \ followed by n instead of a "newline".  Is that really what 
you wanted?

text_new=replace(text_old, chr(10), nl)

This use of replace() is deprecated.  Use text_old.replace() instead.

> It works.

For some definitions of "works".  This only works if you wanted the 
*single* character represented by '\x0a' to turn into the *two* 
characters backslash-followed-by-letter-n.  The following you might find 
instructive:

 >>> old = 'some\x0atext'
 >>> old
'some\ntext'
 >>> print old
some
text
 >>> old.encode('string-escape')
'some\\ntext'
 >>> print old.encode('string-escape')
some\ntext

This will turn other control characters into their equivalent 
two-character escaped representation as well, such as \t and \r, as 
necessary.

-Peter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Writing portable applications

[Mike Meyer]

Ulrich Hobelmann <[EMAIL PROTECTED]> writes:
> Mike Meyer wrote:
>> I'm still waiting for an answer to that one - where's the Java toolkit
>> that handles full-featured GUIs as well as character cell
>> interfaces. Without that, you aren't doing the job that the web
>> technologies do.
> Where is the text-mode browser that would even run part of the web
> apps I use, like home-banking, all web forums, server configuration
> interfaces, etc.?  I think we should leave both these questions
> open. (In fact, as to UIs using Java, blech!  Haven't seen a really
> good one...)

If your web apps are well-written, any of them should work. As
previously stated, Sturgeon's law applies to the web, so chances are
good they aren't well-written.

> I just wonder who wrote fully standards compliant web browsers for
> those 90 platforms.
 Nobody. I doubt there's a fully standards compliant web browser
>>> Nobody, huh?  Then how could you run just ANY web application on those
>>> platforms?
>> The same way you write POSIX applications in the face of buggy
>> implementations - by working around the bugs in the working part of
>> the implementation, and using conditional code where that makes a
>> serious difference.
> But as soon as some user of platform 54 tries your website, she'll
> encounter some weird behavior without even knowing why.  And maybe so
> will you, especially if you don't have that platform there for
> testing. I don't understand how this web thing changes anything...

The only difference is that the user of Platform 54 has a chance to
use your app. Sure, it may not work because that platforms bugs are
radically different from the bugs in the platforms you tested
on. Raising the possibility of your app working from "no way in hell"
to "maybe" is significant.

> With POSIX at least you have a real bug-report for the guy responsible
> for it.  If a platform keeps being buggy, with no fixes coming, screw
> them.  Every user will see that sooner or later, and these platforms
> die.  Even Windows is quite stable/reliable after 10+ years NT!

You do? All your porters reliably give you bug reports? Can I have
some?

 available for *any* platform, much less any non-trivial collection of
 them. You write portable web applications to the standards, and design
 them to degrade gracefully. Then you go back and work around any new
>>> Oh right, they degrade gracefully.  So without Javascript or cookies
>>> (the former is often not implemented) you get a HTML page with an
>>> error notice -- if you're lucky.
>> You left off the important part of what I had to say - that the
>> application be written by a moderately competent web author.
> But if you can cater for all kinds of sub-platforms, then why not just
> provide a CLI as well as those GUI interfaces, when we're duplicating
> work to begin with? ;)

Because I'd rather not duplicate the work.

>>> A server AND client for a simple protocol designed for its task
>>> (i.e. not FTP for instance) can be implemented in much less work than
>>> even designing even part of a web application backend that does that
>>> kind of stuff.
>> Well, if it that easy (and web applications are dead simple), it
>> should be done fairly frequently. Care to provide an example?
>
> We have all the web standards, with various extensions over the
> years. Some FTP clients even don't crash if they see that some server
> doesn't yet support the extension from RFC [EMAIL PROTECTED]  Then there's
> tons of inter-application traffic in XML already, growing fast.  Then
> there are s-expressions (Lisp XML if you want).  Then probably
> thousands of ad-hoc line-based text protocols, but I don't know how
> well they can be extended.  There's CORBA.  Most web standards are
> simple, at least if you would subtract the weird stuff (and IMHO there
> should be new versions of everything with the crap removed).  XML is
> somewhat simple, just hook libxml.

These don't answer the question. Maybe because I didn't explain it
fully. Do you have an example of an application that implements a
simple protocol along with a client and server where HTTP+HTML were
considered as an alternative, and rejected as "more difficult" than
the path actually chosen?

> There's NNTP.  There's RSS.  There's Atom.  The latter two emerged
> quite painlessly, even though you could maybe use some website for
> what they provide.  But this way you have lots of clients for lots of
> platforms already.

NNTP predates HTTP. Atom (and I assume RSS) uses HTTP as a transport,
so there's no new protocol invovled - just a new file format.

 You think you're done. A lot of developers think you can stop with
 the
 first one or two. You're all right for some applications. For others,
 you're not.  Personally, I like applications that run on all the
 platforms I use - and your set doesn't cover all three of those
 systems.
>>> Ok, I'd be interested to hear what those are.  VMS, RiscOS, Mac OS 9...?
>

Re: OpenSource documentation problems

[Mike Meyer]

"Adriaan Renting" <[EMAIL PROTECTED]> writes:
> Marked -1 Flamebait, but I'll respond anyway.

Yup.

> I do agree that a lot of OSS projects seem to lack somewhat in the
> documentation department, compared to a lot of commercial software.

You know what? My experience is just the opposite. Commercial software
(at lest for Windows and OS X end user applications) often comes with
*no* documentation whatsoever. The few exceptions I've encountered
seldom installed it - if you wanted to look at it, you had to get the
distribution media and copy it by hand.

Instead, you get online help. While the latter is valuable, it's not a
substitute for good documentation.

 http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Using select on a unix command in lieu of signal

[rh0dium]

So here's how I solved this..  It's seems crude - but hey it works.
select not needed..

def runCmd( self, cmd, timeout=None ):
self.logger.debug("Initializing function %s - %s" %
(sys._getframe().f_code.co_name,cmd) )
command = cmd + "\n"

child = popen2.Popen3(command)
t0 = time.time()

out = None
while time.time() - t0 < timeout:
if child.poll() != -1:
self.logger.debug("Command %s completed succesfully" %
cmd )
out = child.poll()
results = "".join(child.fromchild.readlines())
results = results.rstrip()
break
print "Still waiting..", child.poll(), time.time() -
t0, t0
time.sleep(.5)

if out == None:
self.logger.warning( "Command: %s failed!" % cmd)
kill = os.kill(child.pid,9)
self.logger.debug( "Killing command %s - Result: %s" %
(cmd, kill))
out = results = None

else:

self.logger.debug("Exit: %s Reullts: %s" % (out,results))

child.tochild.close()
child.fromchild.close()
return out,results

Comments..




rh0dium wrote:
> Paul Rubin wrote:
> > "rh0dium" <[EMAIL PROTECTED]> writes:
> > > Thanks much - Alternatively if anyone else has a better way to do what
> > > I am trying to get done always looking for better ways.  I still want
> > > this to work though..
> >
> > You don't have to use select, since you can use timeouts with normal
> > socket i/o.  So you could use threads.  3000 threads is a lot but not
> > insanely so.
>
> OK I could use the timeout.. but I am using a queue as well.  So each
> thread gets several commands.  I assumed (could be wrong) that if I use
> a timeout the whole thread gets killed not the individual process.  The
> purpose of the queue was to limit the number of concurrent workers, and
> keep the load on the machine somewaht manageable.
>
> So to add more to this here is how I call the runCmd
>
> # Initialize a que to 25 max hosts
> workQ = Queue.Queue(25)
>
> # Start some threads..
> for i in range(MAX_THREADS):
> getReachableHosts(queue=workQ).start()
>
> # Now give the threads something to do..  The nice thing here is
> that by
> # waiting unil now this will hold up the queue..
> for host in base_hosts:
> workQ.put(host)
>
> # After this is finally done thow a null to close the threads..
> for i in range(MAX_THREADS):
> workQ.put(None)
>
> And then getReachables..
>
> class getReachableHosts(threading.Thread):
> def __init__(self,queue=None, ):
> self.logger = logging.getLogger("metriX.%s" %
> self.__class__.__name__)
> self.logger.info("Initializing class %s" %
> self.__class__.__name__)
> self.__queue = queue
> threading.Thread.__init__(self)
>
> def run(self):
> self.logger.debug("Initializing function %s" %
> sys._getframe().f_code.co_name )
> while 1:
> host = self.__queue.get(timeout=5)
> if host is None:
> break
>
> self.logger.debug("Getting open ports on %s" % host)
> command = "nmap -p 22,514 -oG - %s | perl -lane 'print
> unless /^#/'" % host
>
> (out,results)=self.runCmd(cmd=cmd,timeout=5)
> 
> 
> Much appreciate the advice and help!!

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: new line

[Kuljo]

Kuljo wrote:

> Dear friends
> I'm so sorry to bore you with this trivial problem. Allthou: I have string
> having 0x0a as new line, but I should have \n instead.
> How should I solve it?
> I've tried
text_new=tex_old.replace(str(0x0a), '\n')
> and other things, but none of them worked.
> Thanks in advance


I have found this in the meantime:

>>>nl="\\"+"n"
>>>text_new=replace(text_old, chr(10), nl)

It works.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: overload builtin operator

[Bengt Richter]

On Sun, 28 Aug 2005 23:27:07 -0400, "Terry Reedy" <[EMAIL PROTECTED]> wrote:

>I suspect that PyPy, when further alone, will make it easier to do things 
>like develop a customized interpreter that has alternate definitions for 
>builtin operators.  So maybe the OP should ask again in a year or two.
>
OTOH, did you notice that I provided (although I didn't post the underlying
walker framework) a way (that works now) of importing a module with
the / operator effectively converted to a safediv call, as the OP apparently 
wanted?
;-)
BTW, I posted an update that handles non-simple-name-target /= augassigns as 
well,
but though it shows on google groups, I can't see it yet. Still some news server
problem, apparently. Or maybe it's just slogging to catch up after previous 
problem...

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

python-dev Summary for 2005-08-01 through 2005-08-15

[Tony Meyer]

[The HTML version of this Summary is available at
http://www.python.org/dev/summary/2005-08-01_2005-08-15.html]



=
Announcements
=


QOTF: Quote of the Fortnight


Some wise words from Donovan Baarda in the PEP 347 discussions:

It is true that some well designed/developed software becomes reliable
very quickly. However, it still takes heavy use over time to prove that.

Contributing thread:

- `PEP: Migrating the Python CVS to Subversion
`__

[SJB]


Process PEPs


The PEP editors have introduced a new PEP category: "Process", for PEPs that
don't fit into the "Standards Track" and "Informational" categories.  More
detail can be found in `PEP 1`_, which is itself a Process PEP.

.. _PEP 1: http://www.python.org/peps/pep-0001.html

Contributing thread:

- `new PEP type: Process
`__

[TAM]

---
Tentative Schedule for 2.4.2 and 2.5a1 Releases
---

Python 2.4.2 is tentatively scheduled for a mid-to-late September release
and a first alpha of Python 2.5 for March 2006 (with a final release around
May/June).  This means that a PEP for the 2.5 release, detailing what will
be included, will likely be created soon; at present there are various
accepted PEPs that have not yet been implemented.

Contributing thread:

- `plans for 2.4.2 and 2.5a1
`__

[TAM]

=
Summaries
=

---
Moving Python CVS to Subversion
---

The `PEP 347`_ discussion from last fortnight continued this week, with a
revision of the PEP, and a lot more discussion about possible version
control software (RCS) for the Python repository, and where the repository
should be hosted.  Note that this is not a discussion about bug trackers,
which will remain with Sourceforge (unless a separate PEP is developed for
moving that).

Many revision control systems were extensively discussed, including
`Subversion`_ (SVN), `Perforce`_, `Mercurial`_, and `Monotone`_.  Whichever
system is moved to, it should be able to be hosted somewhere (if
*.python.org, then it needs to be easily installable), needs to have
software available to convert a repository from CVS, and ideally would be
open-source; similarity to CVS is also an advantage in that it requires a
smaller learning curve for existing developers.  While Martin isn't willing
to discuss every system there is, he will investigate those that make him
curious, and will add other people's submissions to the PEP, where
appropriate.

The thread included a short discussion about the authentication mechanism
that svn.python.org will use; svn+ssh seems to be a clear winner, and a test
repository will be setup by Martin next fortnight.

The possibility of moving to a distributed revision control system
(particularly `Bazaar-NG`_) was also brought up.  Many people liked the idea
of using a distributed revision control system, but it seems unlikely that
Bazaar-NG is mature enough to be used for the main Python repository at the
current time (a move to it at a later time is possible, but outside the
scope of the PEP).  Distributed RCS are meant to reduce the barrier to
participation (anyone can create their own branches, for example); Bazaar-NG
is also implemented in Python, which is of some benefit.  James Y Knight
pointed out `svk`_, which lets developers create their own branches within
SVN.

In general, the python-dev crowd is in favour of moving to SVN.  Initial
concern about the demands on the volunteer admins should the repository be
hosted at svn.python.org were addressed by Barry Warsaw, who believes that
the load will be easily managed with the existing volunteers.  Various
alternative hosts were discussed, and if detailed reports about any of them
are created, these can be added to the PEP.

While the fate of all PEPS lie with the BDFL (Guido), it is likely that the
preferences of those that frequently check in changes, the pydotorg admins,
and the release managers (who have all given favourable reports so far),
will have a significant effect on the pronouncement of this PEP.

.. _PEP 347: http://www.python.org/peps/pep-0347.html
.. _svk: http://svk.elixus.org/
.. _Perforce: http://www.perforce.com/
.. _Subversion: http://subversion.tigris.org/
.. _Monotone: http://venge.net/monotone/
.. _Bazaar-NG: http://www.bazaar-ng.org/
.. _Mercurial: http://www.selenic.com/mercurial/

Contributing threads:

- `PEP: Migrating the Python CVS to Subversion
`__
- `PEP 347: Migration to Subversion
`__
- `Hosting svn.python.org

Re: using common lisp with python.

[Mike Meyer]

"[EMAIL PROTECTED]" <[EMAIL PROTECTED]> writes:
> Your best bet is probably to look into your LISP environment's FFI
> (Foreign Function Interface).  Most LISP environments have some way to
> call C code directly.  Insofar as going back the other way... that I'm
> a little more sketchy on.  Guile (the Scheme compiler from GNU) is a
> strong contender, though.  It's not Common LISP, but it's a LISP with
> copious documentation for how to call it from C.

I believe SCM is also extensible/embeddable - and is generally a more
complete LISP system than Guile. However, it's still Scheme instead of
CL.

> I really can't see a reason to use Python as a glue layer.  I'd
> recommend rewriting your LISP code in Python before I'd recommend using
> Python to interface between Common LISP and C.

Agreed.

  http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: global interpreter lock

[Mike Meyer]

Bryan Olson <[EMAIL PROTECTED]> writes:
> Mike Meyer wrote:
>  > Bryan Olson writes:
>  > phil hunt wrote:
>  >> > What's important is *predictability*, e.g. which instruction will
>  >> > the computer execute next?
>  >> > If you only have one thread, you can tell by looking at the code
>  >> > what gets executed next. It's very simple.
>  >>Not really. Trivially, an 'if' statement that depends upon input
>  >>data is statically predictable. Use of async I/O means makes the
>  >>programs execution dependent upon external timing.
>  > Yes, but that depenency is tied to a single point - the select
>  > call. The paths after that are statically predictable. This makes the
>  > code very managable.
> Wow -- I could not disagree more. Returning back to some single
> point for every possibly-blocking operation is painful to manage
> even for simple GUIs, and humanly intractable for sophisticated
> services.

I'd be interested in what you're trying to do that winds up as
unmanagable. There are clearly things select+async IO is unsuitable
for. You may be running into problems because you're trying to use it
in such an environment. For instance, it's not clear to me that it
will work well for any kind of GUI programming, though I've had good
look with it for command line interfaces.

> Select is certainly useful, but it scales badly and isn't as
> general as better tools.

It can't take advantage of multiple CPUs. I've not run into scaling
problems on single-CPU systems.

>  > [...] I'm calling the tools available in most programming
>  > languages for dealing with it primitive.
>  > We need better tools.
> Agreed, but if 'select' is someone's idea of the state of the
> art, they have little clue as to the tools already available.

Well, share! If you know of tools that make dealing with concurrent
code more manageable than an async I/O loop and have fewer
restrictions, I'm certainly interested in hearing about them.

> Bringing the tools to Python remains a bit of challenge, largely
> because so many Pythoners are unaware.

Well, the way to fix that is to talk about them!

http://www.mired.org/home/mwm/
Independent WWW/Perforce/FreeBSD/Unix consultant, email for more information.
-- 
http://mail.python.org/mailman/listinfo/python-list

File parser

[Angelic Devil]

I'm building a file parser but I have a problem I'm not sure how to
solve.  The files this will parse have the potential to be huge
(multiple GBs).  There are distinct sections of the file that I
want to read into separate dictionaries to perform different
operations on.  Each section has specific begin and end statements
like the following:

KEYWORD
.
.
.
END KEYWORD

The very first thing I do is read the entire file contents into a
string.  I then store the contents in a list, splitting on line ends
as follows:


file_lines = file_contents.split('\n')


Next, I build smaller lists from the different sections using the
begin and end keywords:


begin_index = file_lines.index(begin_keyword)
end_index = file_lines.index(end_keyword)
small_list = [ file_lines[begin_index + 1] : file_lines[end_index - 1] ]


I then plan on parsing each list to build the different dictionaries.
The problem is that one begin statement is a substring of another
begin statement as in the following example:


BAR
END BAR

FOOBAR
END FOOBAR


I can't just look for the line in the list that contains BAR because
FOOBAR might come first in the list.  My list would then look like

[foobar_1, foobar_2, ..., foobar_n, ..., bar_1, bar_2, ..., bar_m]

I don't really want to use regular expressions, but I don't see a way
to get around this without doing so.  Does anyone have any suggestions
on how to accomplish this? If regexps are the way to go, is there an
efficient way to parse the contents of a potentially large list using
regular expressions?

Any help is appreciated!

Thanks,
Aaron

-- 
"Tis better to be silent and be thought a fool, than to speak and
 remove all doubt."
-- Abraham Lincoln
-- 
http://mail.python.org/mailman/listinfo/python-list

Drag&Drop in wxListCtrl

[[EMAIL PROTECTED]]

Hi,

 I have to create a ListBox in which I have to move items up and down
using DnD. How to create drag and drop call backs in wxListCtrl.
I can see EVT_LIST_BEGIN_DRAG but I could not find any
EVT_LIST_END_DRAG.
So, how can I achieve DnD with ListCtrl?

Thanks in advance for the suggestions.

Regards
Kalyan

-- 
http://mail.python.org/mailman/listinfo/python-list

a list of Checkbuttons

[William Gill]

OK I'm tired, I've got a cold, and my brain isn't working very well.  I 
have a result set ( a tuple of tuples) from a db.  Each element has two 
elements; classification number, and classification heading.  i.e. 
result=((001,'heading one'),(002,'heading two'),...)

classification numbers may not be in numerical order, and may skip 
numbers, so they are not good as an index.

I want to create tkinter Checkbuttons that allow me to select which 
apply to the record I'm editing.  What comes to my sub-par mind is to 
create a list and assign Checkbuttons to each element.  something like:

classifications = []
for count, classification in enumerate(result):
classifications.append( (IntVar(),classification[0]) )
Checkbutton(master,
text=str(classification[0])+': '+classification[1] ,
variable=classifications[-1]).grid(row=count)



then to collect which boxes are checked:

for check in classifications:
 if check[0].get() == 1:
print 'classification number %s selected"  % check[1][0]


Is this the smartest way to do this?  It should work, but will I be able 
to understand this 6 months from now?

Thanks,

Bill
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Jargons of Info Tech industry

[Alan Balmer]

On 29 Aug 2005 21:12:13 GMT, John Bokma <[EMAIL PROTECTED]> wrote:

>> Now, go away. And please, stay away.
>
>Like I already said, it doesn't work that way.

Goodbye, John. Filters set.
-- 
Al Balmer
Balmer Consulting
[EMAIL PROTECTED]
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: trictionary?

[Bengt Richter]

On 28 Aug 2005 18:54:40 -0700, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]> wrote:

>Steven Bethard wrote:
>> Adam Tomjack wrote:
>> > Steven Bethard wrote:
>> > ...
>> >> Using a two element list to store a pair of counts has a bad code
>> >> smell to me.
>> > ...
>> >
>> > Why is that?
>>
>> Note that "code smell"[1] doesn't mean that something is actually wrong,
>> just that it might be.  In Python, pairs are usually handled with
>> tuples[2], but tuples would be inconvenient in this case, since the
>> first value must be modified.  Declaring a class with two attributes as
>> you suggested is often a good substitute, but if the OP's code is really
>> what it looks like, I get another code smell because declaring a class
>> to be used by only 10 lines of code seems like overkill.
>>
>> I also get a code smell from a dict holding two-element lists because
>> I've been writing in Python and answering questions on the Python-list
>> for a couple of years now, and I've never needed one yet. ;-)
>>
>> STeVe
>>
>> [1]http://en.wikipedia.org/wiki/Code_smell
>> [2]http://www.python.org/doc/faq/general.html#why-are-there-separate-tuple-and-list-data-types
>
>Could you do me a favor and see what this smells like?
>
>I put some data in a csv file (with headers) and this will
>bring it in quite simply as a dictionary with [names] as
>keys and an attribute dictionary as the value.
>
>py_monsters.csv:
>
>name,hardiness,agility,friend,courage,room,weight,special_def_odds,armor,weapon,odds,dice,side,hits,reaction,desc
>PIRATE,5,20,0,10,26,300,0,0,11,60,1,10,0,"not met",You see a man with a
>beard and a brass ring in his ear.  He is wearing clothes  made of silk
>and is wielding a very fancily engraved sword.
>
>
>import csv
>temp1 = []
>temp2 = []
>reader = csv.reader(file(r"py_monsters.csv"))
>for rec in reader:
>   temp1.append(rec)
>for i in temp1[1:]:
>   temp2.append((i[0],dict(zip(temp1[0][1:],i[1:]
>monsters = dict(temp2)
>
>This gives me what I want
>
>[('PIRATE', {'reaction': 'not met',
>'agility': '20',
>'room': '26',
>'weight': '300',
>'armor': '0',
>'weapon': '11',
>'hits': '0',
>'side': '10',
>'special_def_odds': '0',
>'courage': '10',
>'hardiness': '5',
>'desc': 'You see a man with a beard and a brass ring in his ear.
>He is wearing clothes  made of silk and is wielding a very fancily
>engraved sword.',
>'odds': '60',
>'friend': '0',
>'dice': '1'})]
>
>so that I can now write code like
>
>if monsters['PIRATE']['reaction']=='not met':
>print monsters['PIRATE']['desc']

I think if the field names are legal python names, I would rather write that 
'if ...'
without all that line noise ;-) E.g., using simple classes (and effectively 
using
their attribute instance dicts essentially the way you used raw dicts), you 
could write

if monsters.PIRATE.reaction == 'not met':
print monsters.PIRATE.desc

Also, if the reaction, agility, etc list is fixed, you could use __slots__ for
better efficiency. Also, your __init__ method for the latter could convert
strings to ints for handier use later (unless csv already does that in the mode
you are using).

>From the above code I infer that you can do define the requisite classes, so
I'll leave it to you ;-) Note that attribute access opens the door to having
default values as class variables, and using properties to retrieve dynamically
calculated values by name. If you are not familiar with those aspects of 
classes,
your progammer.agility value can increase markedly with a little study ;-)

E.g., a name like monsters.random_elf could live right
alongside PIRATE and return one a randomly selected elf from a an internal list
of elves or via random selection from a list of elf names defined at the same
level as PIRATE, etc. etc. You could also dynamically configure these guys
according to distance to food and time since last meal, and if you are
carrying food, etc.

With suitable classes you could put instances in various "spaces" defined
by other classes, that define geometric or social or other interactions.

>
>instead of using stupid index numbers.
>
>But the loader seems to have a kind of perl-like odor to it,
>i.e., next week I won't understand what it does.
>
If you have to do something tricky, choose names wisely and
comment the non-obvious ;-)

Regards,
Bengt Richter
-- 
http://mail.python.org/mailman/listinfo/python-list

networking

[amf]

I'm seeking learn more about Python, and specifically to talk with
people in the greater Boston, MA area. Can anyone suggest the best way
to do this?  Feel free to contact me offline: [EMAIL PROTECTED]

Amf

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: socket.gaierror from httplib

[spamsink42]

fantastic.  many thanks.

regards,
eric

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: python and ajax

[Jp Calderone]

On Mon, 29 Aug 2005 12:04:46 -0700 (PDT), Steve Young <[EMAIL PROTECTED]> wrote:
>Hi, I was wondering if anybody knew of any good
>tutorial/example of AJAX/xmlhttprequest in python.
>Thanks.
>

There's a short example of Nevow's LivePage online here: 
http://divmod.org/svn/Nevow/trunk/examples/livepage/livepage.py

It's not a tutorial by itself, but if you poke around some of the other 
examples and read http://divmod.org/projects/nevow and some of the documents it 
references, you should be able to figure things out.

Jp
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Basic Server/Client socket pair not working

[Jp Calderone]

On Mon, 29 Aug 2005 21:30:51 +0200, Michael Goettsche <[EMAIL PROTECTED]> wrote:
>Hi there,
>
>I'm trying to write a simple server/client example. The client should be able
>to send text to the server and the server should distribute the text to all
>connected clients. However, it seems that only the first entered text is sent
>and received. When I then get prompted for input again and press return,
>nothing gets back to me. Any hints on what I have done would be very much
>appreciated!

You aren't handling readiness notification properly.  Twisted is a good way to 
not have to deal with all the niggling details of readiness notification.  I 
haven't trotted out this example in a while, and it's /almost/ appropriate here 
;) so...

http://www.livejournal.com/users/jcalderone/2660.html

Of course, that's highly obscure and not typical of a Twisted application.  
Nevertheless, I believe it satisfies your requirements.  You might want to take 
a look at http://twistedmatrix.com/projects/core/documentation/howto/index.html 
for a gentler introduction, though.  In particular, the client and server 
HOWTOs.

I'll comment on your code below.

>
>Here's my code:
>
> SERVER ##
>import socket
>import select
>
>mySocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
>mySocket.bind(('', 1))
>mySocket.listen(1)
>
>clientlist = []
>
>while True:
>   connection, details = mySocket.accept()
>   print 'We have opened a connection with', details
>   clientlist.append(connection)
>   readable = select.select(clientlist, [], [])
>   msg = ''
>   for i in readable[0]:

You'll need to monitor sockets for disconnection.  The way you do this varies 
between platforms.  On POSIX, a socket will become readable but reading from it 
will return ''.  On Win32, a socket will show up in the exceptions set (which 
you are not populating).  With the below loop, your server will go into an 
infinite loop whenever a client disconnects, because the chunk will be empty 
and the message will never get long enough to break the loop.

>  while len(msg) < 1024:
> chunk = i.recv(1024 - len(msg))
> msg = msg + chunk

Additionally, select() has only told you that at least /one/ recv() call will 
return without blocking.  Since you call recv() repeatedly, each time through 
the loop after the first has the potential to block indefinitely, only 
returning when more bytes manage to arrive from the client.  This is most 
likely the cause of the freezes you are seeing.  Instead, call recv() only once 
and buffer up to 1024 (if this is really necessary - I do not think it is) over 
multiple iterations through the outermost loop (the "while True:" loop).

>
>   for i in clientlist:
>  totalsent = 0
>  while totalsent < 1024:
> sent = i.send(msg)
> totalsent = totalsent + sent

This isn't quite right either - though it's close.  You correctly monitor how 
much of the message you have sent to the client, since send() may not send the 
entire thing.  However there are two problems.  One is trivial, and probably 
just a think-o: each time through the loop, you re-send `msg', when you 
probably meant to send `msg[totalsent:]' to avoid duplicating the beginning of 
the message and losing the end of it.  The other problem is that the send() 
call may block.  You need to monitor each socket for write-readiness (the 2nd 
group of sockets to select()) and only call send() once for each time a socket 
appears as writable.

>
>## CLIENT 
>import socket
>import select
>
>socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
>socket.connect(("127.0.0.1", 1))
>
>while True:
>text = raw_input("Du bist an der Reihe")
>text = text + ((1024 - len(text)) * ".")
>totalsent = 0
>while totalsent < len(text):
>sent = socket.send(text)
>totalsent = totalsent + sent

Similar problem here as in the server-side send() loop - 
`send(text[totalsent:])' instead of sending just `text'.

>
>msg = ''
>while len(msg) < 1024:
>chunk = socket.recv(1024 - len(msg))
>msg = msg + chunk

This is problematic too, since it means you will only be able to send one 
message for each message received from the server, and vice versa.  Most chat 
sessions don't play out like this.

>
>print msg

I encourage you to take a look at Twisted.  It takes care of all these little 
details in a cross-platform manner, allowing you to focus on your unique 
application logic, rather than solving the same boring problems that so many 
programmers before you have solved.

Hope this helps,

Jp
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: SocketServer and a Java applet listener

[google]

Steve Horsley schreef:

> [EMAIL PROTECTED] wrote:
> > Steve Horsley schreef:
> >
> >
> >>Probably the same problem. If you didn't send a 2 byte length
> >>indicator first, then java's readUTF() will have tried to
> >>interpret the first 2 bytes that you did actually send as the
> >>string length, and may well simply be waiting patiently for the
> >>rest to arrive.
> >>
> >
> > I just couldn't get read/writeUTF and python unicode to interface, so I
> > refactored the applet's socketlistener to convert the
> > socket.getInputStream to a BufferedInputReader on which I call the
> > readline() method.


> There are two normal ways to delineate messages in the
> byte-stream: An explicit length indication up front (which java
> read/writeUTF chooses), or a unique end-of-message indication at
> the end such as your readline() for strings that end in linefeed.
>



>
> If you choose to go for the java read/writeUTF approach, the
> 2-byte length indicator goes Most Significant Byte first, so a
> 100 char string would be preceded by 00 64 ... Also, the
> indicator gives the number of bytes after encoding, not the
> number of characters before encoding.
>

You are right, Steven. I invested some time and isolated the problem of
transferring UTF-8 strings between a python SocketServer and a Java
applet. In a very, very draft version, hereby surrounded by every
disclaimer imagineable (as in "Don't try this at home!") I have put the
result on a webpage:
http://www.phaedro.com/javapythonutf8/

Not only I tried to give a more or less 'generic' solution to the UTF-8
interface (reversing the leading bytes using python's struct module,
and based on SocketServer.StreamingRequestHandler), maybe the
draft-craft helps others looking for (rather scarce) examples of
SocketServer implementations - this one is very trivial so maybe others
can learn more efficiently than I had to do.

-- 
Thijs Cobben
Explicit typing sux.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Cygwin font problems

[Steve Holden]

Robin Becker wrote:
> Steve Holden wrote:
> 
>>Is anyone aware of (a fix for) problems I'm having getting PIL 1.1.5 to 
>>create bitmaps using TrueType and openType fonts? When I create an image 
>>using the standard PIL fonts everything seems fine, but when I use 
>>ImageFont.truetype(, ) no text is drawn.
>>
>>setup.py reports from debug print statements that it's finding freetype21.
>>
>>The same code using (Fredrik's pre-built binary) on Windows runs 
>>correctly and produces correct images.
>>
>>Cygwin runs Python 2.4.1, Windows runs 2.4. Any light on this mystery 
>>gratefully received.
>>
>>regards
>> Steve
> 
> Could PIL have been compiled without freetype support perhaps?
> 
Don't think so, as I don't see any errors in the build, and the closing 
summary from "python setup.py build" reads


PIL 1.1.5 BUILD SUMMARY

version   1.1.5
platform  cygwin 2.4.1 (#1, May 27 2005, 18:02:40)
   [GCC 3.3.3 (cygwin special)]

--- TKINTER support ok
--- JPEG support ok
--- ZLIB (PNG/ZIP) support ok
--- FREETYPE2 support ok


Strangely enough all the tests pass, too!

regards
  Steve
-- 
Steve Holden   +44 150 684 7255  +1 800 494 3119
Holden Web LLC http://www.holdenweb.com/

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Bug in string.find; was: Re: Proposed PEP: New style indexing, was Re: Bug in slice type

[Steve Holden]

Antoon Pardon wrote:
> Op 2005-08-27, Steve Holden schreef <[EMAIL PROTECTED]>:
> 
>>>
>>If you want an exception from your code when 'w' isn't in the string you 
>>should consider using index() rather than find.
> 
> 
> Sometimes it is convenient to have the exception thrown at a later
> time.
> 
> 
>>Otherwise, whatever find() returns you will have to have an "if" in 
>>there to handle the not-found case.
> 
> 
> And maybe the more convenient place for this "if" is in a whole different
> part of your program, a part where using -1 as an invalid index isn't
> at all obvious.
> 
> 
>>This just sounds like whining to me. If you want to catch errors, use a 
>>function that will raise an exception rather than relying on the 
>>invalidity of the result.
> 
> 
> You always seem to look at such things in a very narrow scope. You never
> seem to consider that various parts of a program have to work together.
> 
Or perhaps it's just that I try not to mix parts inappropriately.

> So what happens if you have a module that is collecting string-index
> pair, colleted from various other parts. In one part you
> want to select the last letter, so you pythonically choose -1 as
> index. In an other part you get a result of find and are happy
> with -1 as an indictation for an invalid index. Then these
> data meet.
> 
That's when debugging has to start. Mixing data of such types is 
somewhat inadvisable, don't you agree?

I suppose I can't deny that people do things like that, myself included, 
but mixing data sets where -1 is variously an error flag and a valid 
index is only going to lead to trouble when the combined data is used.

regards
  Steve
-- 
Steve Holden   +44 150 684 7255  +1 800 494 3119
Holden Web LLC http://www.holdenweb.com/

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: socket.gaierror from httplib

[Steve Holden]

[EMAIL PROTECTED] wrote:
> this code
> 
> h=httplib.HTTPConnection('euronext.com')
> h.request('GET',
> 'http://www.euronext.com/home/0,3766,1732,00.html')
> 
> fails with this message
> 
>   File "httplib.py", line 532, in connect
> socket.SOCK_STREAM):
> socket.gaierror: (-2, 'Name or service not known')
> 
> what am i doing wrong?
> 
> thanks
> eric
> 
The HTTPConnection specifies the server you are connection to (you only 
give to domain, which isn't guaranteed to resolve to the same IP 
address, or indeed even any IP address).

Having connected, all you need to present in the request() method call 
is the HTTP method and the URI relative to the server. This works for me:

  >>> import httplib; h=httplib.HTTPConnection('www.euronext.com')
  >>> h.request('GET','/home/0,3766,1732,00.html')
  >>>

regards
  Steve
-- 
Steve Holden   +44 150 684 7255  +1 800 494 3119
Holden Web LLC http://www.holdenweb.com/

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: python and ajax

[Peter Hansen]

Steve Young wrote:
> Hi, I was wondering if anybody knew of any good
> tutorial/example of AJAX/xmlhttprequest in python.
> Thanks.

I can't say if it's "good" since I haven't looked at it yet, but it's 
certainly timely: this was just posted to the c.l.p.announce group:


A simple server which enables Python regular expression tests in a 
webbrowser. Uses SimpleHTTPServer and AJAX.

You need: Python, a modern webbrowser like Firefox, IE (from 5.5), 
Safari) which handles XMLHttpRequests. Currently works best with 
Firefox, any feedback is welcome.

download

retest-0.3.zip - 050828
 http://cthedot.de/retest/retest-0.3.zip
-


-Peter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Exploring outlook contents

[Peter Hansen]

Subir wrote:
>   I am trying to build an application to explore the contents of an
> outlook .pst files. All the reference that I have seen uses the
> registry to do so. Does any one know any other way of doing this. Also,
> is anyone has any code which does something related to this, please let
> me know.

Are you trying to do this without Outlook installed?  Last time I 
checked, the format was proprietary and rather undocumented, and all 
approaches that I found relied on having Outlook actually installed on 
the machine.

-Peter
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Using select on a unix command in lieu of signal

[rh0dium]

Paul Rubin wrote:
> "rh0dium" <[EMAIL PROTECTED]> writes:
> > Thanks much - Alternatively if anyone else has a better way to do what
> > I am trying to get done always looking for better ways.  I still want
> > this to work though..
>
> You don't have to use select, since you can use timeouts with normal
> socket i/o.  So you could use threads.  3000 threads is a lot but not
> insanely so.

OK I could use the timeout.. but I am using a queue as well.  So each
thread gets several commands.  I assumed (could be wrong) that if I use
a timeout the whole thread gets killed not the individual process.  The
purpose of the queue was to limit the number of concurrent workers, and
keep the load on the machine somewaht manageable.

So to add more to this here is how I call the runCmd

# Initialize a que to 25 max hosts
workQ = Queue.Queue(25)

# Start some threads..
for i in range(MAX_THREADS):
getReachableHosts(queue=workQ).start()

# Now give the threads something to do..  The nice thing here is
that by
# waiting unil now this will hold up the queue..
for host in base_hosts:
workQ.put(host)

# After this is finally done thow a null to close the threads..
for i in range(MAX_THREADS):
workQ.put(None)

And then getReachables..

class getReachableHosts(threading.Thread):
def __init__(self,queue=None, ):
self.logger = logging.getLogger("metriX.%s" %
self.__class__.__name__)
self.logger.info("Initializing class %s" %
self.__class__.__name__)
self.__queue = queue
threading.Thread.__init__(self)

def run(self):
self.logger.debug("Initializing function %s" %
sys._getframe().f_code.co_name )
while 1:
host = self.__queue.get(timeout=5)
if host is None:
break

self.logger.debug("Getting open ports on %s" % host)
command = "nmap -p 22,514 -oG - %s | perl -lane 'print
unless /^#/'" % host

(out,results)=self.runCmd(cmd=cmd,timeout=5)


Much appreciate the advice and help!!

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Jargons of Info Tech industry

[John Bokma]

Alan Balmer <[EMAIL PROTECTED]> wrote:

> On 29 Aug 2005 18:21:12 GMT, John Bokma <[EMAIL PROTECTED]> wrote:
> 
>>Alan Balmer <[EMAIL PROTECTED]> wrote:
>>
>>> Why on earth was this cross-posted to comp.lang.c.? Followups set.
>>
>>Your reply is even more meaningless and more noise compared to the 
>>crosspost. Why? You didn't add anything, you quote an entire message and 
>>you just tweaked the follow up to header in a bad way.
> 
> My aim was simply to get it the hell off c.l.c as quickly as possible.

So you edited that one out, and decided to bother all other groups with 
your ineffective attempt?

> Now, go away. And please, stay away.

Like I already said, it doesn't work that way. The only way to make an 
follow up effective is to write a useful contribution to the thread, and 
set the follow up to the most appropriate group (and keep your fingers 
crossed). Not what you did, moreover you only add noise instead of removing 
it. Ignore the thread, it will end in 2-3 days. If you keep feeding it your 
way it will become more and more off topic and last 5-10 days.

-- 
John   Small Perl scripts: http://johnbokma.com/perl/
   Perl programmer available: http://castleamber.com/
Happy Customers: http://castleamber.com/testimonials.html

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Python-list Digest, Vol 23, Issue 415

[Christopher DeMarco]

On Mon, Aug 29, 2005 at 10:43:46PM +0200, [EMAIL PROTECTED] wrote:

> Date: Mon, 29 Aug 2005 16:32:59 -0400
> To: python-list@python.org
> From: Steve Holden <[EMAIL PROTECTED]>
> Subject: Re: NYC Opening



> >THEY ARE LOCATED IN NEW YORK, THIS IS FULL-TIME ONLY, WILL NOT CONSIDER
> >ANYONE FROM OUTSIDE THE US! THIS TEAM IS AN ELITE TEAM, YOU BETTER BE
> >GOOD

> It seems like this bank expects much more of its programmers than it 
> does of its recruitment consultants ...

They've outsourced most of the work to Nigeria...

PY7HON P.R.O.G.R.A.M.M.E.R.S DELIVERED DISCRETELY TO YOUR DOORSTEP


-- 
Christopher DeMarco <[EMAIL PROTECTED]>
Alephant Systems (http://alephant.net)
PGP public key at http://pgp.alephant.net
+1 412 708 9660
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Using select on a unix command in lieu of signal

[Paul Rubin]

"rh0dium" <[EMAIL PROTECTED]> writes:
> Thanks much - Alternatively if anyone else has a better way to do what
> I am trying to get done always looking for better ways.  I still want
> this to work though..

You don't have to use select, since you can use timeouts with normal
socket i/o.  So you could use threads.  3000 threads is a lot but not
insanely so.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: NYC Opening

[Steve Holden]

Kevin McGann wrote:
> A major Investment Bank is searching for a strong Java or C++ developer that
> can build Pipes to Electronic Exchanges and ECNs. This is a new Internal
> Hedge Fund Business and they are looking for superb C++ or Java programmers
> that can enable them to go live in 1-2 months and begin trading. THIS IS AN
> OPPORTUNITY TO WORK IN THE BUSINESS UNIT OF A MAJOR TOP-TIERED BANK!
> Required Skills:
> -Expert Java or C++
> -Extensive Multi-threading
> -OOA/OOD Must have Design Patterns Experience
> -Must have experience building Pipes/Connectivity Infrastructure/FIX
> -Linux, Unix, SQL/RDBMS
> Eventually this person will learn Stat/Arb business for Equities/FX and will
> get to learn about trading strategies and work closely with Quants/Prop
> Traders.
> THEY ARE LOCATED IN NEW YORK, THIS IS FULL-TIME ONLY, WILL NOT CONSIDER
> ANYONE FROM OUTSIDE THE US! THIS TEAM IS AN ELITE TEAM, YOU BETTER BE
> GOOD
> 
> Also have numerous opportunities so if you program in C++ or Java send me an
> email or give me a call.
> -
> Kevin M McGann
> Senior Technology Recruiter
> Continuity Partners Inc.
> (212) 624-9187
> www.cpi-search.com
> [EMAIL PROTECTED]
> -
> 
> 
It seems like this bank expects much more of its programmers than it 
does of its recruitment consultants ...

regards
  Steve
-- 
Steve Holden   +44 150 684 7255  +1 800 494 3119
Holden Web LLC http://www.holdenweb.com/

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: suggestion for Python graphing package, please

[Robert Kern]

Stewart Midwinter wrote:
> I need a graphing library that I can access from within a Tkinter
> application running on Windows.

http://matplotlib.sourceforge.net

-- 
Robert Kern
[EMAIL PROTECTED]

"In the fields of hell where the grass grows high
 Are the graves of dreams allowed to die."
  -- Richard Harter

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: new line

[Robert Kern]

Kuljo wrote:
> Dear friends
> I'm so sorry to bore you with this trivial problem. Allthou: I have string
> having 0x0a as new line, but I should have \n instead.

In [9]: '\x0a'
Out[9]: '\n'

They're the same thing.

-- 
Robert Kern
[EMAIL PROTECTED]

"In the fields of hell where the grass grows high
 Are the graves of dreams allowed to die."
  -- Richard Harter

-- 
http://mail.python.org/mailman/listinfo/python-list

Using select on a unix command in lieu of signal

[rh0dium]

Hi all,

Another newbie question.  So you can't use signals on threads but you
can use select.  The reason I want to do this in the first place it I
need a timeout.  Fundamentally I want to run a command on another
machine, but I need a timeout.  I have to do this to a LOT of machines
( > 3000 ) and threading becomes necessary for timeliess.  So I created
a function which works with signals ( until you throw threading at it..
) but I can't seem to modify it correctly to use select.  Can some
select ( pun intended ) experts out there point out the error of my
way..

Not working RunCmd using select

def runCmd( self, cmd, timeout=None ):

starttime = time.time()

child = popen2.Popen3(cmd)
child.tochild.write("\n")
child.tochild.close()
child.wait()

results = []
results = "".join(child.fromchild.readlines())

endtime = starttime + timeout

r, w, x = select.select(results, [], [], endtime - time.time())

if len(r) == 0:
# We timed out.
prefix = ("TIMED OUT:" + " " * maxlen)[:maxlen]
sys.stdout.write(prefix)
space = ""
os.kill(child.pid,9)
child.fromchild.close()

return results


Working RunCmd using signal

def handler(self, signum, frame):
self.logger.debug("Signal handler called with signal %s" %
signum)

def runCmd( self, cmd, timeout=None ):
self.logger.debug("Initializing function %s - %s" %
(sys._getframe().f_code.co_name,cmd) )

# Set the signal handler and a 5-second alarm
signal.signal(signal.SIGALRM, self.handler)
signal.alarm(timeout)

try:
child = popen2.Popen3(cmd)
child.tochild.write("y\n")
child.tochild.close()
child.wait()

results = "".join(child.fromchild.readlines())
out = child.fromchild.close()
self.logger.debug("command: %s Status: %s PID: %s " % (cmd,
out, child.pid))

if out is None:
out = 0

except:
self.logger.warning( "command: %s failed!" % cmd)
kill = os.kill(child.pid,9)
self.logger.debug( "Killing command %s - Result: %s" %
(cmd, kill))
out = results = None

signal.alarm(0)  # Disable the alarm

return out,results

Thanks much - Alternatively if anyone else has a better way to do what
I am trying to get done always looking for better ways.  I still want
this to work though..

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: close failed: [Errno 0] No error goes to stdout

[Steve Holden]

Yoav wrote:
> I run a Java command line program. The point is, that it's not the 
> program that output this error message for sure. And I don't expect 
> popen3() to catch and report errors. I just want to keep my screen 
> output clean, and I expect popen3() to run the program and not let 
> anything slip to the screen, after all as I understood it is supposed to 
> capture STDERR and STDOUT, but maybe I didn' t understand it right (it's 
> quite probable). Anyway anyway I can do such a thing?
> 
It would be helpful if you could verify your theory by running the 
program from the command line with standard AND error output 
redirection, verifying that no output at all apears on the console when 
you type

cmd > /tmp/stdout 2>/tmp/stderr

You are correct in supposing that popen3 is supposed to trap all stdout 
and stderr output.

regards
  Steve
-- 
Steve Holden   +44 150 684 7255  +1 800 494 3119
Holden Web LLC http://www.holdenweb.com/

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: new line

[Claudio Grondi]

"Kuljo" <[EMAIL PROTECTED]> schrieb im Newsbeitrag
news:[EMAIL PROTECTED]
> Dear friends
> I'm so sorry to bore you with this trivial problem. Allthou: I have string
> having 0x0a as new line, but I should have \n instead.
> How should I solve it?
> I've tried
> >>>text_new=tex_old.replace(str(0x0a), '\n')
> and other things, but none of them worked.
> Thanks in advance

text_new=tex_old.replace(chr(0x0a), '\r\n')
works for me.

Claudio
P.S. str(0x0a) == '10'


-- 
http://mail.python.org/mailman/listinfo/python-list

suggestion for Python graphing package, please

[Stewart Midwinter]

I need a graphing library that I can access from within a Tkinter application running on Windows.

It needs to be able to draw some *simple* 2D plots, and then output
them to a file (e.g. .PNG, .JPG) for inclusion in a HTML-formatted
e-mail to interested parties. 

Many of the packages that I've looked at a) either don't output to a
file, or b) are very old (3-5 years), or c) don't run on Windows.

I don't actually care if the library itself is written in Python or c++ or something else.
Any suggestions?

thanks,-- Stewart Midwinter[EMAIL PROTECTED][EMAIL PROTECTED]Skype:midtoad  GoogleTalk:midtoad  iChatAV:midtoad
MSN:midtoad  Yahoo:midtoad  AIM:midtoad1
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Robust statistics and optimmization from Python

[Tim Churches]

[EMAIL PROTECTED] wrote:
> Robert Kern wrote:
> 
> 
> 
>>If you find suitable
>>FORTRAN or C code that implements a particular "robust" algorithm, it
>>can probably wrapped for scipy relatively easily.
> 
> 
> An alternative would be to call R (a free statistical package) from
> Python, using something like the R/SPlus - Python Interface at
> http://www.omegahat.org/RSPython/ .

Unless you really want to call Python from R (as opposed to calling R
from Python), I strongly suggest that you use RPy (http://rpy.sf.net)
rather than RSPython. RPy is much easier to install and use and far less
buggy than RSPython.

> Many statistical algorithms,
> including those for robust statistics, have been implemented in R,
> usually by wrapping C or Fortran 77 code.

Yup, and if you really don't like the extra dependency or extra memory
requirements of R and RPy, it is often possible to port the R code back
to Python (or Numeric Python), with some effort.

Tim C

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: python and ajax

[matt]

Steve-

I recently ported version 1.3 of cpaint to python.  Soon after version
2.0 was released and I haven't really looked at it since.  The 1.3
stuff was really simple though if you understand cgi, then you just
implement a endpoint for your request to call.  The javascript side is
really the only thing new (might be a little learning if you having
done much js).

I think that more advanced ajax libraries like dojo or openrico are
probably better suited to more complicated ajax use.  Though they are
more focused on the js frontend stuff.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: python xml DOM? pulldom? SAX?

[William Park]

jog <[EMAIL PROTECTED]> wrote:
> Hi,
> I want to get text out of some nodes of a huge xml file (1,5 GB). The
> architecture of the xml file is something like this
> 
>
> bla
> 
> 
>   
>   blablabla
> 
>
>
>
> 
> 
> I want to combine the text out of page:title and page:revision:text for
> every single page element. One by one I want to index these combined
> texts (so for each page one index)
> What is the most efficient API for that?: SAX ( I don?t thonk so) DOM
> or pulldom?
> Or should I just use Xpath somehow.
> I don`t want to do anything else with his xml file afterwards.
> I hope someone will understand me.
> Thank you very much
> Jog

I would use Expat interface from Python, Awk, or even Bash shell.  I'm
most familiar with shell interface to Expat, which would go something
like

start() # Usage: start tag att=value ...
{
case $1 in
page) unset title text ;;
esac
}
data()  # Usage: data text
{
case ${XML_TAG_STACK[0]}.${XML_TAG_STACK[1]}.${XML_TAG_STACK[2]} in
title.page.*) title=$1 ;;
text.revision.page) text=$1 ;;
esac
}
end()   # Usage: end tag
{
case $1 in
page) echo "title=$title text=$text" ;;
esac
}
expat -s start -d data -e end < file.xml

-- 
William Park <[EMAIL PROTECTED]>, Toronto, Canada
ThinFlash: Linux thin-client on USB key (flash) drive
   http://home.eol.ca/~parkw/thinflash.html
BashDiff: Super Bash shell
  http://freshmeat.net/projects/bashdiff/
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: reg email packages work

[Steve Holden]

praba kar wrote:
> Dear All,
> 
>I am working in web based email system project.
> Here I try to build email message
> from scratch. I used below code to build email
> message
> 
>msg =  email.MIMEBase('text','html')
>msg['Return-Path'] = user+'@'+domain
>msg['Date'] = formatdate(localtime=1)
>msg['Subject'] =  subject
>msg['From'] = from_name
>msg['To'] = to
>msg['Cc'] = cc
>msg.set_payload("Body of the email messagge")
>fh = os.popen('/bin/sendmail  %s'% (eachid),'w')
>fh.write(msg.as_string())
>fh.close()
> 
> This code will build plain email message properly.
> But after building the message.  If a email user
> download this mail through out look express then
> this email message will display without any alignment.
>  If a user type 3 paragraph message
> outlook express display as a single line.  What
> could be problem?.  Kindly let me know ASAP
> 
It's obvious you aren't using that EXACT code, because it doesn't 
formulate a three-paragraph message. So the bit we really need to see is 
how you capture and formulate the argument to set_payload().

regards
  Steve
-- 
Steve Holden   +44 150 684 7255  +1 800 494 3119
Holden Web LLC http://www.holdenweb.com/

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Basic Server/Client socket pair not working

[Nicolas Couture]

Michael Goettsche wrote:
> Hi there,
>
> I'm trying to write a simple server/client example. The client should be able
> to send text to the server and the server should distribute the text to all
> connected clients. However, it seems that only the first entered text is sent
> and received. When I then get prompted for input again and press return,
> nothing gets back to me. Any hints on what I have done would be very much
> appreciated!
>
> Here's my code:
>
>  SERVER ##
> import socket
> import select
>
> mySocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
> mySocket.bind(('', 1))
> mySocket.listen(1)
>
> clientlist = []
>
> while True:
>connection, details = mySocket.accept()
>print 'We have opened a connection with', details
>clientlist.append(connection)
>readable = select.select(clientlist, [], [])
>msg = ''
>for i in readable[0]:

for i in readable:

>   while len(msg) < 1024:
>  chunk = i.recv(1024 - len(msg))
>  msg = msg + chunk
>
>for i in clientlist:
>   totalsent = 0
>   while totalsent < 1024:
>  sent = i.send(msg)
>  totalsent = totalsent + sent
>
> ## CLIENT 
> import socket
> import select
>
> socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
> socket.connect(("127.0.0.1", 1))
>
> while True:
> text = raw_input("Du bist an der Reihe")
> text = text + ((1024 - len(text)) * ".")
> totalsent = 0
> while totalsent < len(text):
> sent = socket.send(text)
> totalsent = totalsent + sent
>
> msg = ''
> while len(msg) < 1024:
> chunk = socket.recv(1024 - len(msg))
> msg = msg + chunk
>  
> print msg

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: NooB Question

[Jeff Schwab]

APCass wrote:
> How do you execute a .py in Linux with KDE?  If I double click on my
> program it opens Kwrite, for editing.

Try inserting this as the first line of the file:

#!/usr/bin/env python
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: global interpreter lock

[Bryan Olson]

Mike Meyer wrote:
 > Bryan Olson writes:
 > phil hunt wrote:
 >
 >> > What's important is *predictability*, e.g. which instruction will
 >> > the computer execute next?
 >> >
 >> > If you only have one thread, you can tell by looking at the code
 >> > what gets executed next. It's very simple.
 >>Not really. Trivially, an 'if' statement that depends upon input
 >>data is statically predictable. Use of async I/O means makes the
 >>programs execution dependent upon external timing.
 >
 >
 > Yes, but that depenency is tied to a single point - the select
 > call. The paths after that are statically predictable. This makes the
 > code very managable.

Wow -- I could not disagree more. Returning back to some single
point for every possibly-blocking operation is painful to manage
even for simple GUIs, and humanly intractable for sophisticated
services.

Select is certainly useful, but it scales badly and isn't as
general as better tools.

 > [...] I'm calling the tools available in most programming
 > languages for dealing with it primitive.
 >
 > We need better tools.

Agreed, but if 'select' is someone's idea of the state of the
art, they have little clue as to the tools already available.
Bringing the tools to Python remains a bit of challenge, largely
because so many Pythoners are unaware.


-- 
--Bryan
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: python image thumbnail generator?

[Damjan]

Chris Dewin wrote:

> Hi. I run a website for my band, and the other guys want an image gallery.
> 
> I'm thinking it would be nice and easy, if we could just upload a jpg into
> a dir called "gallery/". When the client clicks the "gallery" link, a
> cgi script could search the gallery/ dir, and create thumbnails of any
> jpeg images that don't already have a thumbnail associated with them. The
> script could then generate a page of clickable thumbnails.

Once I made an example mod_python handler, that resized images on the fly.
For example:
 http://server/folder/image.jpg - would give you the original image, served
directly by apache without any performance hit.

 http://server/folder/image.jpg?thumbnail  - would resize the picture (and
cache the result on disk) and return that, on a second request it would
return the cached image very fast by calling an apache.send_file function.

see
http://www.modpython.org/pipermail/mod_python/2004-September/016471.html

-- 
damjan
-- 
http://mail.python.org/mailman/listinfo/python-list

new line

[Kuljo]

Dear friends
I'm so sorry to bore you with this trivial problem. Allthou: I have string
having 0x0a as new line, but I should have \n instead.
How should I solve it?
I've tried 
>>>text_new=tex_old.replace(str(0x0a), '\n')
and other things, but none of them worked.
Thanks in advance
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Virtual Slicing

[Bryan Olson]

Sybren Stuvel wrote:
 > Bryan Olson enlightened us with:
 >
 >>I recently wrote a module supporting value-shared slicing.
 >
 > Maybe I'm dumb, but could you explain this concept? Why would someone
 > whant this?

My original motivation was reduce the amount of copying in some
tools that parse nested structures. All I really needed at the
time was a reference to a string, and the start and stop values.
Once I adopted Python's sequence interface, I thought I might as
well implement it consistently, generally, and completely.

So the first reason someone might want this is for efficiency,
in space and/or time.

The second reason is more abstract. Python's slice assignment is
a useful feature, but the slice selection must appear on the
right-hand-side of assignment. VSlice lets one instantiate the
updatable slice as an object, and pass it around.

I looked into supporting slice assignment between slices of
different sizes when possible, but the various options I came up
with all sucked.


-- 
--Bryan
-- 
http://mail.python.org/mailman/listinfo/python-list

Basic Server/Client socket pair not working

[Michael Goettsche]

Hi there,

I'm trying to write a simple server/client example. The client should be able 
to send text to the server and the server should distribute the text to all 
connected clients. However, it seems that only the first entered text is sent 
and received. When I then get prompted for input again and press return, 
nothing gets back to me. Any hints on what I have done would be very much 
appreciated!

Here's my code:

 SERVER ##
import socket
import select
 
mySocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
mySocket.bind(('', 1))
mySocket.listen(1)
 
clientlist = []
 
while True:
   connection, details = mySocket.accept()
   print 'We have opened a connection with', details
   clientlist.append(connection)
   readable = select.select(clientlist, [], [])
   msg = ''
   for i in readable[0]:
  while len(msg) < 1024:
 chunk = i.recv(1024 - len(msg))
 msg = msg + chunk
 
   for i in clientlist:
  totalsent = 0
  while totalsent < 1024:
 sent = i.send(msg)
 totalsent = totalsent + sent
 
## CLIENT 
import socket
import select
 
socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
socket.connect(("127.0.0.1", 1))
 
while True:
text = raw_input("Du bist an der Reihe")
text = text + ((1024 - len(text)) * ".")
totalsent = 0
while totalsent < len(text):
sent = socket.send(text)
totalsent = totalsent + sent
 
msg = ''
while len(msg) < 1024:
chunk = socket.recv(1024 - len(msg))
msg = msg + chunk
 
print msg
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Robust statistics and optimmization from Python

[araki]

use R. it's pretty highend, and there is an interface for python.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: python image thumbnail generator?

[Paul Rubin]

"Fredrik Lundh" <[EMAIL PROTECTED]> writes:
> > You could do it with PIL, or run jpegtran in an external process.
> > jpegtran may be easier.
> 
> eh?  are you sure you know what jpegtran does?
> 
> JPEGTRAN(1) 

Whoops, sorry, right, jpegtran is for rotating the images.  I meant:
use a pipeline like

djpeg -scale 1/4 | cjpeg

That's how I usually do it.  Main disadvantage is the scale factor has
to be 1/2, 1/4, 1/8, etc., not arbitrary amounts like 0.3456.
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: python and ajax

[Do Re Mi chel La Si Do]

Hi !


Here :  http://wikipython.flibuste.net/moin.py/AJAX


@-salutations

Michel Claveau 


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Exploring outlook contents

[Beowulf TrollsHammer]

Subir wrote:

> Hi,
>
>   I am trying to build an application to explore the contents of an
> outlook .pst files. All the reference that I have seen uses the
> registry to do so. Does any one know any other way of doing this. Also,
> is anyone has any code which does something related to this, please let
> me know.
>
> -Subir


Check this out:

http://www.boddie.org.uk/python/COM.html

HTH

-- 
http://mail.python.org/mailman/listinfo/python-list

python and ajax

[Steve Young]

Hi, I was wondering if anybody knew of any good
tutorial/example of AJAX/xmlhttprequest in python.
Thanks.

-Steve




Start your day with Yahoo! - make it your home page 
http://www.yahoo.com/r/hs 
 
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: python xml DOM? pulldom? SAX?

[Alan Kennedy]

[Alan Kennedy]
 >>SAX is perfect for the job. See code below.

[Fredrik Lundh]
 > depends on your definition of perfect...

Obviously, perfect is the eye of the beholder ;-)

[Fredrik Lundh]
> using a 20 MB version of jog's sample, and having replaced
> the print statements with local variable assignments, I get the
> following timings:
> 
> 5 lines of cElementTree code: 7.2 seconds
> 60+ lines of xml.sax code: 63 seconds
> 
> (Python 2.4.1, Windows XP, Pentium 3 GHz)

Impressive!

At first, I thought your code sample was building a tree for the entire 
document, so I checked the API docs. It appeared to me that an event 
processing model *couldn't* obtain the text for the node when notified 
of the node: the text node is still in the future.

That's when I understood the nature of iterparse, which must generate an 
event *after* the node is complete, and it's subdocument reified. That's 
also when I understood the meaning of the "elem.clear()" call at the 
end. Only the required section of the tree is modelled in memory at any 
given time. Nice.

There are some minor inefficiencies in my pure python sax code, e.g. 
building the stack expression for every evaluation, but I left them in 
for didactic reasons. But even if every possible speed optimisation was 
added to my python code, I doubt it would be able to match your code.

I'm guessing that a lot of the reason why cElementTree performs best is 
because the model-building is primarily implemented in C: Both of our 
solutions run python code for every node in the tree, i.e. are O(N). But 
yours also avoids the overhead of having function-calls/stack-frames for 
every single node event, by processing all events inside a single function.

If the SAX algorithm were implemented in C (or Java) for that matter, I 
wonder if it might give comparable performance to the cElementTree code, 
primarily because the data structures it is building are simpler, 
compared to the tree-subsections being reified and discarded by 
cElementTree. But that's not of relevance, because we're looking for 
python solutions. (Aside: I can't wait to run my solution on a 
fully-optimising PyPy :-)

That's another nice thing I didn't know (c)ElementTree could do.

enlightened-ly'yrs,

-- 
alan kennedy
--
email alan:  http://xhaus.com/contact/alan
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Cygwin font problems

[Robin Becker]

Steve Holden wrote:
> Is anyone aware of (a fix for) problems I'm having getting PIL 1.1.5 to 
> create bitmaps using TrueType and openType fonts? When I create an image 
> using the standard PIL fonts everything seems fine, but when I use 
> ImageFont.truetype(, ) no text is drawn.
> 
> setup.py reports from debug print statements that it's finding freetype21.
> 
> The same code using (Fredrik's pre-built binary) on Windows runs 
> correctly and produces correct images.
> 
> Cygwin runs Python 2.4.1, Windows runs 2.4. Any light on this mystery 
> gratefully received.
> 
> regards
>  Steve
Could PIL have been compiled without freetype support perhaps?

-- 
Robin Becker
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: python xml DOM? pulldom? SAX?

[Fredrik Lundh]

Alan Kennedy wrote:

> SAX is perfect for the job. See code below.

depends on your definition of perfect...

using a 20 MB version of jog's sample, and having replaced
the print statements with local variable assignments, I get the
following timings:

5 lines of cElementTree code: 7.2 seconds
60+ lines of xml.sax code: 63 seconds

(Python 2.4.1, Windows XP, Pentium 3 GHz)

 



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Jargons of Info Tech industry

[John Bokma]

Alan Balmer <[EMAIL PROTECTED]> wrote:

> On Fri, 26 Aug 2005 16:47:10 GMT, Chris Head <[EMAIL PROTECTED]>
> wrote:
> 
>>This point I agree with. There are some situations - 'net cafes included
>>- - where thick e-mail clients don't work. Even so, see below.
> 
> I use Portable Thunderbird, on a USB memory stick. All I need is a USB
> port and an internet connection.

It's a brave internetcafe that allows such things. (I mean technically one 
can have a Portable Spam Device :-).

-- 
John   Small Perl scripts: http://johnbokma.com/perl/
   Perl programmer available: http://castleamber.com/
Happy Customers: http://castleamber.com/testimonials.html

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Jargons of Info Tech industry

[John Bokma]

Alan Balmer <[EMAIL PROTECTED]> wrote:

> Why on earth was this cross-posted to comp.lang.c.? Followups set.

Your reply is even more meaningless and more noise compared to the 
crosspost. Why? You didn't add anything, you quote an entire message and 
you just tweaked the follow up to header in a bad way.

-- 
John   Small Perl scripts: http://johnbokma.com/perl/
   Perl programmer available: http://castleamber.com/
Happy Customers: http://castleamber.com/testimonials.html

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: using common lisp with python.

[Robert Kern]

[EMAIL PROTECTED] wrote:
> Your best bet is probably to look into your LISP environment's FFI
> (Foreign Function Interface).  Most LISP environments have some way to
> call C code directly.  Insofar as going back the other way... that I'm
> a little more sketchy on.  Guile (the Scheme compiler from GNU) is a
> strong contender, though.  It's not Common LISP, but it's a LISP with
> copious documentation for how to call it from C.

ECL might be a good choice for real Common Lisp (more or less).

http://ecls.sourceforge.net/

> I really can't see a reason to use Python as a glue layer.  I'd
> recommend rewriting your LISP code in Python before I'd recommend using
> Python to interface between Common LISP and C.

Agreed.

-- 
Robert Kern
[EMAIL PROTECTED]

"In the fields of hell where the grass grows high
 Are the graves of dreams allowed to die."
  -- Richard Harter

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: using common lisp with python.

[[EMAIL PROTECTED]]

Your best bet is probably to look into your LISP environment's FFI
(Foreign Function Interface).  Most LISP environments have some way to
call C code directly.  Insofar as going back the other way... that I'm
a little more sketchy on.  Guile (the Scheme compiler from GNU) is a
strong contender, though.  It's not Common LISP, but it's a LISP with
copious documentation for how to call it from C.

I really can't see a reason to use Python as a glue layer.  I'd
recommend rewriting your LISP code in Python before I'd recommend using
Python to interface between Common LISP and C.

-- 
http://mail.python.org/mailman/listinfo/python-list

command history behavior in windows and linux

[Leo]

Does someone know if there is a setting in the python Unix world to
make the command history behave as it does in the Windows intepreter?
Specifically, I like the fact that the command history remembers which
of a sequence of commands is the one that I last issued. For example,
suppose that I typed the following lines into the interpreter:

import foo
for e in foo.container:
   print e

In Unix, I would have to tap up arrow 3 times for each line:

up up up gives me "import foo". I press enter.
up up up gives me "for...". I press enter.
up up up gives me "   print...". I press enter.

In Windows, I get this behavior I like better:

up up up gives me "import foo". I press enter.
down gives me "for...". I press enter.
down gives me "   print...". I press enter.

How do I get the second behavior to take place in all the platforms I
use? Also, the windows version remembers my commands across sections.
How do I enable that in the python version?

I tried editing the inputrc file that controls the readline library by
adding the following lines:

$if python
set history-preserve-point on
$endif

but this did not seem to work.

TIA,
Leo.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Newbie question: Sub-interpreters for CAD program

[David MacQuigg]

On 27 Aug 2005 17:00:07 -0700, "sonicSpammersGoToHellSmooth"
<[EMAIL PROTECTED]> wrote:

>Cool, I went to the UofA for my MS in ECE, 2000.  I did my theses under
>Chuck Higgins. --
>http://neuromorph.ece.arizona.edu/pubs/ma_schwager_msthesis.pdf
>
>The tools we had were constantly underwhelming me, so I've been
>thinking for years that a properly designed new toolset for students
>should be marketable, etc.  I'll take a look at your site (although I
>think I may have come across it before.)

A toolset for students is exactly what we need.  Whether that turns
into a marketable product is a question for later.  At this point, we
need a few people with a strong motivation to help students.  Sounds
like you might have what it takes.  Send me an email if you are
interested.

-- Dave

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Newbie question: Sub-interpreters for CAD program

[David MacQuigg]

On Sat, 27 Aug 2005 16:56:03 -0500, Terry Hancock
<[EMAIL PROTECTED]> wrote:
>On Saturday 27 August 2005 03:21 am, David MacQuigg wrote:

>> There is a similar lack of interest in the academic community.  None
>> of this is likely to lead to publications in scholarly journals.
>
>I'm confused by what you meant by this.  Are you saying that academics
>are afraid of using or creating open source CAD tools, or that they have
>a lack of interest in tools development, because it won't generate papers
>(directly anyway)?

It seems like a lack of interest in tools development, because there
are no new fundamental principles, sophisticated math, or anything
that could help directly and in the short term to get a publication.
There is probably also a perception, shared with engineers in
industry, that the complexity of these tools is inherent in the task.
It's OK for a full-time engineer to spend a few months learning the
intricacies of a poorly-designed scripting language that works with
just one tool, but not appropriate for students.

My hope is that we can get a few good projects to demonstrate the
utility of Python in doing sophisticated designs with simple tools.
Then we will have a foothold in the Universities.  Next will be small
companies that can't afford a CAD department with 10 engineers
dedicated to tool setup.

--
Dave

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: python xml DOM? pulldom? SAX?

[Alan Kennedy]

[jog]
 > I want to get text out of some nodes of a huge xml file (1,5 GB). The
 > architecture of the xml file is something like this

[snip]

 > I want to combine the text out of page:title and page:revision:text
 > for every single page element. One by one I want to index these
 > combined texts (so for each page one index)
 > What is the most efficient API for that?:
 > SAX ( I don´t thonk so)

SAX is perfect for the job. See code below.

 > DOM

If your XML file is 1.5G, you'll need *lots* of RAM and virtual memory 
to load it into a DOM.

 > or pulldom?

Not sure how pulldom does it's pull "optimizations", but I think it 
still builds an in-memory object structure for your document, which will 
still take buckets of memory for such a big document. I could be wrong 
though.

 > Or should I just use Xpath somehow.

Using xpath normally requires building a (D)OM, which will consume 
*lots* of memory for your document, regardless of how efficient the OM is.

Best to use SAX and XPATH-style expressions.

You can get a limited subset of xpath using a SAX handler and a stack. 
Your problem is particularly well suited to that kind of solution. Code 
that does a basic job of this for your specific problem is given below.

Note that there are a number of caveats with this code

1. characterdata handlers may get called multiple times for a single xml 
text() node. This is permitted in the SAX spec, and is basically a 
consequence of using buffered IO to read the contents of the xml file, 
e.g. the start of a text node is at the end of the last buffer read, and 
the rest of the text node is at the beginning of the next buffer.

2. This code assumes that your "revision/text" nodes do not contain 
mixed content, i.e. a mixture of elements and text, e.g. 
"This is a piece of revision 
text. The below code will fail to extract all 
character data in that case.

#-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=
import xml.sax

class Page:

   def append(self, field_name, new_value):
 old_value = ""
 if hasattr(self, field_name):
   old_value = getattr(self, field_name)
 setattr(self, field_name, "%s%s" % (old_value, new_value))

class page_matcher(xml.sax.handler.ContentHandler):

   def __init__(self, page_handler=None):
 xml.sax.handler.ContentHandler.__init__(self)
 self.page_handler = page_handler
 self.stack = []

   def check_stack(self):
 stack_expr = "/" + "/".join(self.stack)
 if '/parent/page' == stack_expr:
   self.page = Page()
 elif '/parent/page/title/text()' == stack_expr:
   self.page.append('title', self.chardata)
 elif '/parent/page/revision/id/text()' == stack_expr:
   self.page.append('revision_id', self.chardata)
 elif '/parent/page/revision/text/text()' == stack_expr:
   self.page.append('revision_text', self.chardata)
 else:
   pass

   def startElement(self, elemname, attrs):
 self.stack.append(elemname)
 self.check_stack()

   def endElement(self, elemname):
 if elemname == 'page' and self.page_handler:
   self.page_handler(self.page)
   self.page = None
 self.stack.pop()

   def characters(self, data):
 self.chardata = data
 self.stack.append('text()')
 self.check_stack()
 self.stack.pop()

testdoc = """

 
  Page number 1
  p1
  
r1
revision one
  
 
 
  Page number 2
  p2
  
r2
revision two
  
 

"""

def page_handler(new_page):
   print "New page"
   print "title\t\t%s" % new_page.title
   print "revision_id\t%s" % new_page.revision_id
   print "revision_text\t%s" % new_page.revision_text
   print

if __name__ == "__main__":
   parser = xml.sax.make_parser()
   parser.setContentHandler(page_matcher(page_handler))
   parser.setFeature(xml.sax.handler.feature_namespaces, 0)
   parser.feed(testdoc)
#-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

HTH,

-- 
alan kennedy
--
email alan:  http://xhaus.com/contact/alan
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: trictionary?

[Randy Bush]

> So I'm going to try to pump you for a little more information here.  Is 
> your goal to count, for each week, how many times it's "full" and how 
> many times it's "not full"?  What do you use the counts for?  What does 
> "full" mean?  Is it always a 0 or 1?  What's the importance of the 
> output formatting?

'full' is boolean.  it says whether a particular bgp announcement
was for the entire ip address allocation, or is a longer prefix.
e.g., if an allocation was for 666.42.0.0/16 and we heard a bgp
announcement for 666.42.1.0/24 that is !full, while an announcement
for the prefix 666.42.0.0/16 is full.

you asked :-)

>  for start, end, AS, full in heard:
>  week = int((start-startDate)/aWeek)
>  if week in bin:
>  bin[week][not full] += 1
>  else:
>  # I'm assuming "full" takes the values 0 or 1
>  # but if not, you can coerce it with bool()
>  bin[week] = [full, int(not full)]

hmmm.  this also reads well.

as an old pascal and modula-2 bondage and discipline type, i gotta
say is it a breath of fresh air to be in a language and community
which care about how code reads more than how clever it is.

randy

-- 
http://mail.python.org/mailman/listinfo/python-list

Exploring outlook contents

[Subir]

Hi,

  I am trying to build an application to explore the contents of an
outlook .pst files. All the reference that I have seen uses the
registry to do so. Does any one know any other way of doing this. Also,
is anyone has any code which does something related to this, please let
me know.

-Subir

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Jargons of Info Tech industry

[Alan Balmer]

On Fri, 26 Aug 2005 16:47:10 GMT, Chris Head <[EMAIL PROTECTED]>
wrote:

>This point I agree with. There are some situations - 'net cafes included
>- - where thick e-mail clients don't work. Even so, see below.

I use Portable Thunderbird, on a USB memory stick. All I need is a USB
port and an internet connection.
-- 
Al Balmer
Balmer Consulting
[EMAIL PROTECTED]
-- 
http://mail.python.org/mailman/listinfo/python-list

ANN: PyDev 0.9.8 released

[Fabio Zadrozny]

Hi All,

PyDev - Python IDE (Python Development Enviroment for Eclipse) version 
0.9.8 has just been released.

Check the homepage (http://pydev.sourceforge.net/) for more details.

Details for Release: 0.9.8

Major highlights:
---

* Jython integrated.
* Jython debugger support added.

Others that are new and noteworthy:
-

* jython integration supports spaces for jython.jar and java install
* jython code-completion support for new style objects (jython 
2.2a1) has been enhanced.
* many templates were added
* the grammar evolved a lot, so, now you actually have decorators in 
the grammar, list comprehension on method calls and tuples and the new 
from xxx import (a,b,c) syntax.
* pylint supports spaces
* pylint is no longer distributed with pydev (it must be installed 
in the site-packages and its location must be specified in the preferences)
* some problems regarding 'zombie processes' after eclipse exit with 
the shells used for code-completion should be fixed

Special thanks
---

This release would not be possible without help from:

OctetString, for the financial support for making jython support possible!
Vitor Oba for debugger patches!

Cheers,

Fabio

-- 
Fabio Zadrozny
--
Software Developer

ESSS - Engineering Simulation and Scientific Software
www.esss.com.br

PyDev - Python Development Enviroment for Eclipse
pydev.sf.net
pydev.blogspot.com


-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Bug in string.find; was: Re: Proposed PEP: New style indexing,was Re: Bug in slice type

[Steven Bethard]

Antoon Pardon wrote:
> I think a properly implented find is better than an index.

See the current thread in python-dev[1], which proposes a new method, 
str.partition().  I believe that Raymond Hettinger has shown that almost 
all uses of str.find() can be more clearly be represented with his 
proposed function.

STeVe

[1]http://mail.python.org/pipermail/python-dev/2005-August/055781.html
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: GIL, threads and scheduling - performance cost

[Pierre Barbier de Reuille]

[EMAIL PROTECTED] a écrit :
> Merci Pierre,
> 
> Yes I agree and plan to move more to C/C++ and releasing the GIL when
> entering C/C++.
> 
> I also need to understand my original question re GIL and rescheduling.
> I fear that lock/unlock too often is also causing delays due to context
> switching.

Well, concerning GIL and rescheduling, releasing a lock is very likely
to induce rescheduling and context switch, but it won't if the system
considers it worthless. But this is true that lock/unlock has a cost and
so, doing it too much will slow down your app.

> 
> BTW do you have any hints/comments on SWIG/BOOST etc to glue PY and
> C/C++ ?

Well, for C I would recommand SWIG. For C++, I personnaly use
Boost.Python. However, you must know that SWIG did not support C++ well
when I had to choose between the two, so there was no question for me.
Then, somehow it depends on how you feel with the tools: SWIG depends on
a specific language and has its own tools while Boost is entirely
written in C++ and uses heavily templates. For performances, Boost
generate better code that SWIG, mainly because SWIG relies on Python
code that encapsulate the actual C functions while Boost does everything
in C++. Also, at least with G++, compiling Boost extension is very time-
and memory-consuming due to its heavy use of templates. As I don't use
SWIG, I don't know about the community, but at least for Boost.Python it
is quite active and questions on the mailing list are answered quickly
enough. Well, this is very quick but if you need more detailed
information, I recommend you to visit both websites.

> 
> 
> Alan
> 

Pierre
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: trictionary?

[Randy Bush]

> bin = {}
> for start, end, AS, full in heard:
>week = int((start-startDate)/aWeek)
>counters = bin.setdefault(week, [0, 0])
>if full:
>   counters[0] += 1
>else:
>   counters[1] += 1

yes!  thanks!

> Using an idea you used earlier, you could get smaller code by saying:
> for start, end, AS, full in heard:
>week = int((start-startDate)/aWeek)
>counters = bin.setdefault(week, [0, 0])
>counters[not full] += 1
> Or
> for start, end, AS, full in heard:
>week = int((start-startDate)/aWeek)
>bin.setdefault(week, [0, 0])[not full] += 1
> Or even
> for start, end, AS, full in heard:
>   bin.setdefault(int((start-startDate)/aWeek), [0, 0])[not full] += 1

as you say, too clever.

> Using lists to represent structs is perfectly fine if the list doesn't 
> live longer than about one screen of code.

i can definitely see that.  in last weeks installment, i buried a
complex trinary tree in a class.

thanks for the advice!

randy

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: python xml DOM? pulldom? SAX?

[Fredrik Lundh]

"jog" wrote:

> I want to get text out of some nodes of a huge xml file (1,5 GB). The
> architecture of the xml file is something like this

> I want to combine the text out of page:title and page:revision:text for
> every single page element. One by one I want to index these combined
> texts (so for each page one index)

here's one way to do it:

try:
import cElementTree as ET
except ImportError:
from elementtree import ElementTree as ET

for event, elem in ET.iterparse(file):
if elem.tag == "page":
title = elem.findtext("title")
revision = elem.findtext("revision/text")
print title, revision
elem.clear() # won't need this any more

references:

http://effbot.org/zone/element-index.htm
http://effbot.org/zone/celementtree.htm (for best performance)
http://effbot.org/zone/element-iterparse.htm

 



-- 
http://mail.python.org/mailman/listinfo/python-list

Re: python xml DOM? pulldom? SAX?

[Michael Ekstrand]

On 29 Aug 2005 08:17:04 -0700
"jog" <[EMAIL PROTECTED]> wrote:
> I want to get text out of some nodes of a huge xml file (1,5 GB). The
> architecture of the xml file is something like this
> [structure snipped]
> I want to combine the text out of page:title and page:revision:text
> for every single page element. One by one I want to index these
> combined texts (so for each page one index)
> What is the most efficient API for that?: SAX ( I don´t thonk so) DOM
> or pulldom?

Definitely SAX IMHO, or xml.parsers.expat. For what you're doing, an
event-driven interface is ideal. DOM parses the *entire* XML tree into
memory at once, before you can do anything - highly inefficient for a
large data set like this. I've never used pulldom, it might have
potential, but from my (limited and flawed) understanding of it, I
think it may also wind up loading most of the file into memory by the
time you're done.

SAX will not build any memory structures other than the ones you
explicitly create (SAX is commonly used to build DOM trees). With SAX,
you can just watch for any tags of interest (and perhaps some
surrounding tags to provide context), extract the desired data, and all
that very efficiently.

It took me a bit to get the hang of SAX, but once I did, I haven't
looked back. Event-driven parsing is a brilliant solution to this
problem domain.

> Or should I just use Xpath somehow.

XPath usually requires a DOM tree on which it can operate. The Python
XPath implementation (in PyXML) requires DOM objects. I see this as
being a highly inefficient solution.

Another potential solution, if the data file has extraneous
information: run the source file through an XSLT transform that strips
it down to only the data you need, and then apply SAX to parse it.

- Michael
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Robust statistics and optimmization from Python

[beliavsky]

Robert Kern wrote:



> If you find suitable
> FORTRAN or C code that implements a particular "robust" algorithm, it
> can probably wrapped for scipy relatively easily.

An alternative would be to call R (a free statistical package) from
Python, using something like the R/SPlus - Python Interface at
http://www.omegahat.org/RSPython/ . Many statistical algorithms,
including those for robust statistics, have been implemented in R,
usually by wrapping C or Fortran 77 code.

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: python xml DOM? pulldom? SAX?

[tooper]

Hi,

I'd advocate for using SAX, as DOM related methods implies loading the
complete XML content in memory whereas SAX grab things on the fly.
SAX method should therefore be faster and less memory consuming...

By the way, if your goal is to just "combine the text out of page:title
and page:revision:text for every single page element", maybe you should
also consider an XSLT filter.

Regards,
Thierry

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Jargons of Info Tech industry

[Alan Balmer]

Why on earth was this cross-posted to comp.lang.c.? Followups set.

On Mon, 29 Aug 2005 12:26:11 GMT, [EMAIL PROTECTED] wrote:

>In comp.lang.perl.misc Mike Meyer <[EMAIL PROTECTED]> wrote:
>> [EMAIL PROTECTED] writes:
>>> In comp.lang.perl.misc John Bokma <[EMAIL PROTECTED]> wrote:
 Chris Head <[EMAIL PROTECTED]> wrote:
> What advantages would those be (other than access from 'net cafes, but
> see below)?
 And workplaces. Some people have more then one computer in the house. My 
 partner can check her email when I had her over the computer. When I 
 want to check my email when she is using it, I have to change the 
 session, fire up Thunderbird (which eats away 20M), and change the 
 session back.
>>> Not a Windows solution, but I find the 'screen' utility invaluable as
>>> I can have my email, news, and an editor open in different screens
>>> and then when I need to move to a different machine, I can simply
>>> detach and reattach screen without disturbing anything that
>>> might be running.
> 
>> For a more  portable solution, check out VNC.
> 
>I know... but it is a bugger to set up and I believe it is no longer
>freeware (if it ever was), and it does not have the stark simplicity
>which screen has... I only need to have a compiled version of screen
>on the machine on which I do most of my work and be able to ssh/telnet
>to that machine without involving any additional software installations
>on other machines.
>
>Axel
-- 
Al Balmer
Balmer Consulting
[EMAIL PROTECTED]
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: GIL, threads and scheduling - performance cost

[adsheehan]

Merci Pierre,

Yes I agree and plan to move more to C/C++ and releasing the GIL when
entering C/C++.

I also need to understand my original question re GIL and rescheduling.
I fear that lock/unlock too often is also causing delays due to context
switching.

BTW do you have any hints/comments on SWIG/BOOST etc to glue PY and
C/C++ ?


Alan

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: GIL, threads and scheduling - performance cost

[Pierre Barbier de Reuille]

[EMAIL PROTECTED] a écrit :
> Hi all,
> 
> Wondering if a GIL lock/unlock causes a re-schedule/contect swap when
> embedding Python in a multi-threaded C/C++ app on Unix ?
> 
> If so, do I have any control or influence on this re-scheduling ?
> 
> The app suffers from serious performance degradation (compared to pure
> c/C++) and high context switches that I suspect the GIL unlocking may
> be aggravating ?

Well, where do you observe this degradation ? When replacing part of the
C++ code by Python's code ? Or on C++ code running parallel to your
Python code ?

Because in the first case, well, this is just something natural ! Python
runtime overhead is much greater than C++ because of its dynamic nature
(it has to resolve all the symbols at runtime ...). And given the policy
for locking/releasing the GIL, I doubt it has serious performance issues
compared to the Python interpreter itself. If current performance is an
issue, consider implementing more in C/C++ ! This will be mainly true if
you currently have some heavy looping in Python. Python is very neat to
put together processor-intensive functions written in other languages,
but not to implement them. (as an exemple looh at that:
http://www.python.org/doc/essays/list2str.html )

> 
> Thanks for any help.
> 
> Alan
> 

Pierre
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: trictionary?

[Steven Bethard]

Randy Bush wrote:
> Steven Bethard wrote:
>> It would probably help if you explained what the real problem is
>> you're trying to solve.
> 
> actually, that code fragment was meant to do that.  it's pretty much
> what i needed to do at that point, just the variable names made
> simple.

Yeah, I gathered that.  Sometimes though, you may not ever need to get 
to "that point" if the surrounding code can be properly reorganized. 
But I can't tell if that's possible unless you explain what the goal of 
the program is, not just how your current code tries to approach that goal.

So I'm going to try to pump you for a little more information here.  Is 
your goal to count, for each week, how many times it's "full" and how 
many times it's "not full"?  What do you use the counts for?  What does 
"full" mean?  Is it always a 0 or 1?  What's the importance of the 
output formatting?

> so, to do this using the real names, it looks like
> 
>for [start, end, AS, full] in heard:
>   week = int((start-startDate)/aWeek)
>   if week in bin:
>  if full:
>   bin[week][0] += 1
>else:
>   bin[week][1] += 1
>   else:
>  if full:
>   bin[week] = [1, 0]
>else:
>   bin[week] = [0, 1]
>...
>for i, (j, k) in bin.iteritems():
>   if j == 0:
>print str(i) + ",," + str(k)
>   elif k == 0:
>print str(i) + "," + str(j)
>   else:
>print str(i) + "," + str(j) + "," + str(k)

For the current code, I'd probably go with something like:

 for start, end, AS, full in heard:
 week = int((start-startDate)/aWeek)
 if week in bin:
 bin[week][not full] += 1
 else:
 # I'm assuming "full" takes the values 0 or 1
 # but if not, you can coerce it with bool()
 bin[week] = [full, int(not full)]
...
for i, (j, k) in bin.iteritems():
result = [str(i), j and str(j) or '']
# special-case k because if it's zero, the reference
# code drops a comma
if k:
result.append(str(k))
print ','.join(result)

But if you're just trying to count the number of times a week is full or 
not full, I'd probably do something like:

 for start, end, AS, full in heard:
 week = int((start-startDate)/aWeek)
 if week in bin:
 bin[week].append(full)
 else:
 bin[week] = [full]
...
for i, fulls in bin.iteritems():
j = sum(fulls)
k = len(fulls) - j
...


STeVe
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: using common lisp with python.

[[EMAIL PROTECTED]]

basically, what I'm looking to do is use python as a bridge between C
and Common Lisp to create a virtual city that contains Artificial life.

-- 
http://mail.python.org/mailman/listinfo/python-list

socket.gaierror from httplib

[spamsink42]

this code

h=httplib.HTTPConnection('euronext.com')
h.request('GET',
'http://www.euronext.com/home/0,3766,1732,00.html')

fails with this message

  File "httplib.py", line 532, in connect
socket.SOCK_STREAM):
socket.gaierror: (-2, 'Name or service not known')

what am i doing wrong?

thanks
eric

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: Jargons of Info Tech industry

[T Beck]

John Bokma wrote:
> "T Beck" <[EMAIL PROTECTED]> wrote:
>
> >
[snip]
> > alongside of it.  The internet is a free-flowing evolving place... to
> > try to protect one little segment like usenet from ever evolving is
> > just ensuring it's slow death, IMHO.
>
> And if so, who cares? As long as people hang out on Usenet it will stay.
> Does Usenet need al those extra gimmicks? To me, it would be nice if a
> small set would be available. But need? No.
>
> The death of Usenet has been predicted for ages. And I see only more and
> more groups, and maybe more and more people on it.
>
> As long as people who have to say something sensible keep using it, it
> will stay.
>
I suppose I was (as many people on the internet have a bad habit of
doing) being more caustic than was strictly necessary.  I don't really
forsee the death of usenet anytime soon, I just don't think the idea of
it evolving is necessarily bad.  I don't really have alot of vested
interest one way or the other, to be honest, and I'm perfectly happy
with the way it is.

I just think it's a naive view to presume it never will change, because
change is what the internet as a whole was built on.

I think I'll calmly butt out now  ^_^

-- T Beck

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: trictionary?

[Steven Bethard]

[EMAIL PROTECTED] wrote:
> import csv
> temp1 = []
> temp2 = []
> reader = csv.reader(file(r"py_monsters.csv"))
> for rec in reader:
>   temp1.append(rec)
> for i in temp1[1:]:
>   temp2.append((i[0],dict(zip(temp1[0][1:],i[1:]
> monsters = dict(temp2)

I would tend to write this as:

import csv
reader = csv.reader(file(r"py_monsters.csv"))
labels = reader.next()[1:]
monsters = dict((row[0], dict(zip(labels, row[1:])))
 for row in reader)

which I believe gives equivalent output:

py> csv_text = """\
... 
name,hardiness,agility,friend,courage,room,weight,special_def_odds,armor,weapon,odds,dice,side,hits,reaction,desc
... PIRATE,5,20,0,10,26,300,0,0,11,60,1,10,0,"not met",You see a man 
with a beard and a brass ring in his ear.  He is wearing clothes  made 
of silk and is wielding a very fancily engraved sword."""
py> f = StringIO.StringIO(csv_text)
py> reader = csv.reader(f)
py> labels = reader.next()[1:]
py> monsters = dict((row[0], dict(zip(labels, row[1:])))
... for row in reader)
py> monsters
{'PIRATE': {'reaction': 'not met', 'agility': '20', 'room': '26', 
'weight': '300', 'armor': '0', 'weapon': '11', 'hits': '0', 'side': 
'10', 'special_def_odds': '0', 'courage': '10', 'hardiness': '5', 
'desc': 'You see a man with a beard and a brass ring in his ear.  He is 
wearing clothes  made of silk and is wielding a very fancily engraved 
sword.', 'odds': '60', 'friend': '0', 'dice': '1'}}

STeVe
-- 
http://mail.python.org/mailman/listinfo/python-list

Python port on Palm Treo?

[Paul Watson]

Has anyone done or worked on a port of Python to the Treo?
-- 
http://mail.python.org/mailman/listinfo/python-list

Re: overload builtin operator

[Robert Kern]

Iain King wrote:
> Robert Kern wrote:

>>You have silly users.
> 
> You mean you don't?  Damn.  Can I have some of yours?

No, you may not. Mine! All mine!

-- 
Robert Kern
[EMAIL PROTECTED]

"In the fields of hell where the grass grows high
 Are the graves of dreams allowed to die."
  -- Richard Harter

-- 
http://mail.python.org/mailman/listinfo/python-list

Re: trictionary?

[Steven Bethard]

Adam Tomjack wrote:
> I'd write it like this:
> 
>bin = {}
>for start, end, AS, full in heard:
>   week = int((start-startDate)/aWeek)
>   counters = bin.setdefault(week, [0, 0])
>   if full:
>  counters[0] += 1
>   else:
>  counters[1] += 1
> 
>for week, (times_full, times_not_full) in bin.iteritems():
>   print ...
> 
> Seriously, use setdefault() as often as you can.  It may be a little 
> awkward at first, but after a little bit, you instantly know what it 
> means.  It takes out a whole if/else statement which will make your code 
> smaller and more readable at the same time.

However, you should be aware that it can make your code slower.  If the 
default value is expensive to create,

 value = dct.setdefault(key, expensive_func())

will often be slower and/or more memory intensive than

 try:
 value = dct[key]
 except KeyError:
 value = expensive_func()

or

 if key in dct:
 value = dct[key]
 else:
 value = expensive_func()

Personally, I waver back and forth between using setdefault.  It's kind 
of handy, but it never reads quite right.  While I've used it often 
enough to grok it pretty quickly, it's still consumes more of my mental 
processing than some other approaches.

I also tend to agree with the assessment[1] (I believe due to Raymond 
Hettinger) that setdefault is poorly designed, and should instead set a 
default value for the entire dictionary, e.g. so that you could do:

 counts = {}
 counts.setdefault(value=0)
 for elem in data:
 counts[elem] += 1

Unfortunately, this is pretty badly backwards incompatible, so I can 
only really hope for this change in Python 3.0 which is still a number 
of years off.

STeVe

[1]http://wiki.python.org/moin/Python3%2e0Suggestions#head-b384410f8b2dc16fd74c9eec764680e428643d73
-- 
http://mail.python.org/mailman/listinfo/python-list