Re: Recommendation for small, fast, Python based web server
On Dec 9, 4:05 pm, pyt...@bdurham.com wrote: > Daniel, > > > I'm using cherrypy for this purpose, actually together with turbogears 1. > > My research has constantly pointed back to cherrypy as a tool of choice > for building local web servers. My initial impression was that cherrypy > was too big and complicated for my simple task. However, I'm going to > re-examine this assumption and take another look at cherrypy. > > Thanks for your help! > > Regards, > Malcolm tornado all the way, it is teh radness: http://www.tornadoweb.org/ epoll based python server. fun to hack on. def check it out. -- http://mail.python.org/mailman/listinfo/python-list
how to knock a page out of file cache
Can anyone recommend a way to read a file ensuring that it is not coming from file cache on Linux? I'm trying to write a metric script for measuring http connect + read times from a web server over localhost. I want to plot both file cache read times and non-cached files. I thought of simply opening and writing to the file to dirty it's pages, but there no guarantee that pdflush will have already written the dirty pages to disk -pretty sure it depends on all the dirty ratio and intervals. Does anybody know of a system call that will 'knock' the file out of file cache? Can madvise or fadvise do this? I know this is mostly a unix question, but I'm asking in the context of how to do this in python. -- http://mail.python.org/mailman/listinfo/python-list
Re: basic thread question
On Aug 18, 3:18 pm, Derek Martin wrote: > On Tue, Aug 18, 2009 at 03:10:15PM -0500, Derek Martin wrote: > > I have some simple threaded code... If I run this > > with an arg of 1 (start one thread), it pegs one cpu, as I would > > expect. If I run it with an arg of 2 (start 2 threads), it uses both > > CPUs, but utilization of both is less than 50%. Can anyone explain > > why? > > Ah, searching while waiting for an answer (the e-mail gateway is a bit > slow, it seems...) I discovered that the GIL is the culprate. > Evidently this question comes up a lot. It would probably save a lot > of time on the part of those who answer questions here, as well as > those implementing solutions in Python, if whoever is maintaining the > docs these days would put a blurb about this in the docs in big bold > letters... Concurrency being perhaps the primary reason to use > threading, essentially it means that Python is not useful for the > sorts of problems that one would be inclined to solve they way my code > works (or rather, was meant to). It would be very helpful to know > that *before* one tried to implement a solution that way... especially > for solutions significantly less trivial than mine. ;-) > > Thanks > > -- > Derek D. Martinhttp://www.pizzashack.org/ > GPG Key ID: 0x81CFE75D > > application_pgp-signature_part > < 1KViewDownload I would still watch that video which will explain a bit more about the GIL. -- http://mail.python.org/mailman/listinfo/python-list
Re: basic thread question
On Aug 18, 1:10 pm, Derek Martin wrote: > I have some simple threaded code... If I run this > with an arg of 1 (start one thread), it pegs one cpu, as I would > expect. If I run it with an arg of 2 (start 2 threads), it uses both > CPUs, but utilization of both is less than 50%. Can anyone explain > why? > > I do not pretend it's impeccable code, and I'm not looking for a > critiqe of the code per se, excepting the case where what I've written > is actually *wrong*. I hacked this together in a couple of minutes, > with the intent of pegging my CPUs. Performance with two threads is > actually *worse* than with one, which is highly unintuitive. I can > accomplish my goal very easily with bash, but I still want to > understand what's going on here... > > The OS is Linux 2.6.24, on a Ubuntu base. Here's the code: > > Thanks > > -=-=-=-=- > > #!/usr/bin/python > > import thread, sys, time > > def busy(thread): > x=0 > while True: > x+=1 > > if __name__ == '__main__': > try: > cpus = int(sys.argv[1]) > except ValueError: > cpus = 1 > print "cpus = %d, argv[1] = %s\n" % (cpus, sys.argv[1]) > i=0 > thread_list = [] > while i < cpus: > x = thread.start_new_thread(busy, (i,)) > thread_list.append(x) > i+=1 > while True: > pass > > -- > Derek D. Martinhttp://www.pizzashack.org/ > GPG Key ID: 0x81CFE75D > > application_pgp-signature_part > < 1KViewDownload watch this and all your findings will be explained: http://blip.tv/file/2232410 this talk marked a pivotal moment in my understanding of python threads and signal handling in threaded programs. -- http://mail.python.org/mailman/listinfo/python-list
Re: Non-blocking read with popen subprocess
On Jul 30, 7:30 pm, Jonathan Gardner wrote: > On Jul 30, 5:24 pm, Dhanesh wrote: > > > > > how can I we have a non blocking read ? > > Seehttp://docs.python.org/library/popen2.html#flow-control-issues > > Note well: In the non-blocking world, you have to use select() or poll > () to get your job done. I solved a problem just yesterday regarding select on a Popen.stdout file descriptor that was created with a 100Mb buffer. I'd love it if anybody could critique my solution and my understanding of my findings. In my case, select would return with my Popen object ready for read, I'd go off and try to read and then my read would block. After a few strace's I found that Popen.stdout.read(), which I am assuming is the same as the read builtin, would make multiple read system calls, in many cases blocking until it returned my default buffer size. Sure enough, it's in the docs: http://docs.python.org/library/stdtypes.html?highlight=fread """ Note that this method may call the underlying C function fread() more than once in an effort to acquire as close to size bytes as possible. Also note that when in non-blocking mode, less data than was requested may be returned, even if no size parameter was given. """ I couldn't find a way to tell subprocess to return my stdout fd in non- blocking mode, but then I thought about it not being a problem with the fd and how it was opened as much as a problem with the read call. A non-blocking fd will have a read call throw EWOULDBLOCK when asked for more bytes than is in buffer, a blocking fd would have read just return fewer bytes ...do I have this this right? I can handle assembling the data fine, I just don't want to block. I wrote a little test program to spit out random stdout to test a Popen object's stdout in os.read. The len() of the result of os.read was all over the place depending on how quickly i re-ran read() on the fd. There were slight delays from time to time, but I attributed that to the interpreter being context switched in, ssh delays ...or the weather...in general the return was pretty fast. I switched my program to use os.read and now it's snappy and spends most of it's time blocking on select exactly where I want it to. (this is on linux btw, I guess os.read is different everywhere) > You may want to look at "communicate" > (http://docs.python.org/library/subprocess.html#popen-objects) which may be > what you need. -- http://mail.python.org/mailman/listinfo/python-list
interacting with an updatedb generated data file within python
Does anybody have any recommendations on how to interact with the data file that updatedb generates? I'm running through a file list in sqlite that I want to check against the file system. updatedb is pretty optimized for building an index and storing it, but I see no way to query the db file other than calling locate itself. This would require me to fork and exec for every single file I want to verify - I'd be better off doing the stat myself in that case, but I'd really rather let updatedb build the index for me. I searched high and low for any sort of library that is well suited for reading these data files, but I've found nothing for any language other than the source for locate and updatedb itself. -- http://mail.python.org/mailman/listinfo/python-list
Re: Translating pysnmp oids to human readable strings
On Mar 5, 11:22 pm, SpamMePlease PleasePlease
wrote:
> On Fri, Mar 6, 2009 at 1:10 AM, birdsong wrote:
> > On Mar 5, 2:30 pm, SpamMePlease PleasePlease
> > wrote:
> >> On Thu, Mar 5, 2009 at 10:12 PM, birdsong wrote:
> >> > On Mar 5, 1:05 pm, birdsong wrote:
> >> >> On Mar 5, 12:32 pm, SpamMePlease PleasePlease
>
> >> >> wrote:
> >> >> > Hey list,
>
> >> >> > I was given a task, to reproduce functionality of command specified
> >> >> > below by writing proper python functions to reuse in some monitoring
> >> >> > script:
>
> >> >> > rivendell# snmpwalk -Os -mALL -v1 -cgabilgathol 10.0.6.66
> >> >> > .1.3.6.1.4.1.2636.5.1.1.2
> >> >> > jnxBgpM2PeerIdentifier.0.ipv4."".unknown."".0.1.38.101.87.145 =
> >> >> > STRING: 66.250.1.253
> >> >> > jnxBgpM2PeerIdentifier.0.ipv4.38.101.161.119.1.38.101.161.118 =
> >> >> > STRING: 66.28.1.85
> >> >> > jnxBgpM2PeerIdentifier.0.ipv4.64.200.59.74.1.64.200.59.73 = STRING:
> >> >> > 64.200.68.12
> >> >> > jnxBgpM2PeerIdentifier.0.ipv4.72.37.131.250.1.72.37.131.249 = STRING:
> >> >> > 64.235.224.240
> >> >> > jnxBgpM2PeerState.0.ipv4."".unknown."".0.1.38.101.87.145 = INTEGER:
> >> >> > established(6)
> >> >> > jnxBgpM2PeerState.0.ipv4.38.101.161.119.1.38.101.161.118 = INTEGER:
> >> >> > established(6)
> >> >> > jnxBgpM2PeerState.0.ipv4.64.200.59.74.1.64.200.59.73 = INTEGER:
> >> >> > established(6)
> >> >> > jnxBgpM2PeerState.0.ipv4.72.37.131.250.1.72.37.131.249 = INTEGER:
> >> >> > established(6)
> >> >> > (more output)
>
> >> >> > I have already found a pysnmp library to fetch the data from the
> >> >> > device with a minimal amount of code:
>
> >> >> > from pysnmp.entity.rfc3413.oneliner import cmdgen
> >> >> > from pysnmp.smi import *
> >> >> > import string
>
> >> >> > cmdGen = cmdgen.CommandGenerator()
> >> >> > errorIndication, errorStatus, errorIndex, varBinds =
> >> >> > cmdgen.CommandGenerator().nextCmd(cmdgen.CommunityData('AmonMuil',
> >> >> > 'gabilgathol', 0),
> >> >> > cmdgen.UdpTransportTarget(('fw-1.datacenter.gondor.net', 161)),
> >> >> > (1,3,6,1,4,1,2636,5,1,1,2))
>
> >> >> > print errorIndication, errorStatus
> >> >> > for i in varBinds:
> >> >> > print i
>
> >> >> > The problem is that I have completely stuck on the result I am
> >> >> > experiencing being totally human unreadable, like this:
>
> >> >> > rivendell# python snmp.py
> >> >> > None 0
> >> >> > [(ObjectName('1.3.6.1.4.1.2636.5.1.1.2.1.1.1.1.0.1.0.0.0.0.1.38.101.87.145'),
> >> >> > OctetString('B\xfa\x01\xfd'))]
> >> >> > [(ObjectName('1.3.6.1.4.1.2636.5.1.1.2.1.1.1.1.0.1.38.101.161.119.1.38.101.161.118'),
> >> >> > OctetString('B\x1c\x01U'))]
> >> >> > [(ObjectName('1.3.6.1.4.1.2636.5.1.1.2.1.1.1.1.0.1.64.200.59.74.1.64.200.59.73'),
> >> >> > OctetString('@\xc8D\x0c'))]
> >> >> > [(ObjectName('1.3.6.1.4.1.2636.5.1.1.2.1.1.1.1.0.1.72.37.131.250.1.72.37.131.249'),
> >> >> > OctetString('@\xeb\xe0\xf0'))]
> >> >> > [(ObjectName('1.3.6.1.4.1.2636.5.1.1.2.1.1.1.2.0.1.0.0.0.0.1.38.101.87.145'),
> >> >> > Integer32('6'))]
> >> >> > [(ObjectName('1.3.6.1.4.1.2636.5.1.1.2.1.1.1.2.0.1.38.101.161.119.1.38.101.161.118'),
> >> >> > Integer32('6'))]
> >> >> > [(ObjectName('1.3.6.1.4.1.2636.5.1.1.2.1.1.1.2.0.1.64.200.59.74.1.64.200.59.73'),
> >> >> > Integer32('6'))]
> >> >> > [(ObjectName('1.3.6.1.4.1.2636.5.1.1.2.1.1.1.2.0.1.72.37.131.250.1.72.37.131.249'),
> >> >> > Integer32('6'))]
> >> >> > [(ObjectName('1.3.6.1.4.1.2636.5.1.1.2.1.1.1.3.0.1.0.0.0.0.1.38.101.87.145'),
> >> >> > Integer32('2'))]
>
> >> >> > Since I cant find any way to translate these numbers to the same th
Re: Translating pysnmp oids to human readable strings
On Mar 5, 2:30 pm, SpamMePlease PleasePlease
wrote:
> On Thu, Mar 5, 2009 at 10:12 PM, birdsong wrote:
> > On Mar 5, 1:05 pm, birdsong wrote:
> >> On Mar 5, 12:32 pm, SpamMePlease PleasePlease
>
> >> wrote:
> >> > Hey list,
>
> >> > I was given a task, to reproduce functionality of command specified
> >> > below by writing proper python functions to reuse in some monitoring
> >> > script:
>
> >> > rivendell# snmpwalk -Os -mALL -v1 -cgabilgathol 10.0.6.66
> >> > .1.3.6.1.4.1.2636.5.1.1.2
> >> > jnxBgpM2PeerIdentifier.0.ipv4."".unknown."".0.1.38.101.87.145 =
> >> > STRING: 66.250.1.253
> >> > jnxBgpM2PeerIdentifier.0.ipv4.38.101.161.119.1.38.101.161.118 =
> >> > STRING: 66.28.1.85
> >> > jnxBgpM2PeerIdentifier.0.ipv4.64.200.59.74.1.64.200.59.73 = STRING:
> >> > 64.200.68.12
> >> > jnxBgpM2PeerIdentifier.0.ipv4.72.37.131.250.1.72.37.131.249 = STRING:
> >> > 64.235.224.240
> >> > jnxBgpM2PeerState.0.ipv4."".unknown."".0.1.38.101.87.145 = INTEGER:
> >> > established(6)
> >> > jnxBgpM2PeerState.0.ipv4.38.101.161.119.1.38.101.161.118 = INTEGER:
> >> > established(6)
> >> > jnxBgpM2PeerState.0.ipv4.64.200.59.74.1.64.200.59.73 = INTEGER:
> >> > established(6)
> >> > jnxBgpM2PeerState.0.ipv4.72.37.131.250.1.72.37.131.249 = INTEGER:
> >> > established(6)
> >> > (more output)
>
> >> > I have already found a pysnmp library to fetch the data from the
> >> > device with a minimal amount of code:
>
> >> > from pysnmp.entity.rfc3413.oneliner import cmdgen
> >> > from pysnmp.smi import *
> >> > import string
>
> >> > cmdGen = cmdgen.CommandGenerator()
> >> > errorIndication, errorStatus, errorIndex, varBinds =
> >> > cmdgen.CommandGenerator().nextCmd(cmdgen.CommunityData('AmonMuil',
> >> > 'gabilgathol', 0),
> >> > cmdgen.UdpTransportTarget(('fw-1.datacenter.gondor.net', 161)),
> >> > (1,3,6,1,4,1,2636,5,1,1,2))
>
> >> > print errorIndication, errorStatus
> >> > for i in varBinds:
> >> > print i
>
> >> > The problem is that I have completely stuck on the result I am
> >> > experiencing being totally human unreadable, like this:
>
> >> > rivendell# python snmp.py
> >> > None 0
> >> > [(ObjectName('1.3.6.1.4.1.2636.5.1.1.2.1.1.1.1.0.1.0.0.0.0.1.38.101.87.145'),
> >> > OctetString('B\xfa\x01\xfd'))]
> >> > [(ObjectName('1.3.6.1.4.1.2636.5.1.1.2.1.1.1.1.0.1.38.101.161.119.1.38.101.161.118'),
> >> > OctetString('B\x1c\x01U'))]
> >> > [(ObjectName('1.3.6.1.4.1.2636.5.1.1.2.1.1.1.1.0.1.64.200.59.74.1.64.200.59.73'),
> >> > OctetString('@\xc8D\x0c'))]
> >> > [(ObjectName('1.3.6.1.4.1.2636.5.1.1.2.1.1.1.1.0.1.72.37.131.250.1.72.37.131.249'),
> >> > OctetString('@\xeb\xe0\xf0'))]
> >> > [(ObjectName('1.3.6.1.4.1.2636.5.1.1.2.1.1.1.2.0.1.0.0.0.0.1.38.101.87.145'),
> >> > Integer32('6'))]
> >> > [(ObjectName('1.3.6.1.4.1.2636.5.1.1.2.1.1.1.2.0.1.38.101.161.119.1.38.101.161.118'),
> >> > Integer32('6'))]
> >> > [(ObjectName('1.3.6.1.4.1.2636.5.1.1.2.1.1.1.2.0.1.64.200.59.74.1.64.200.59.73'),
> >> > Integer32('6'))]
> >> > [(ObjectName('1.3.6.1.4.1.2636.5.1.1.2.1.1.1.2.0.1.72.37.131.250.1.72.37.131.249'),
> >> > Integer32('6'))]
> >> > [(ObjectName('1.3.6.1.4.1.2636.5.1.1.2.1.1.1.3.0.1.0.0.0.0.1.38.101.87.145'),
> >> > Integer32('2'))]
>
> >> > Since I cant find any way to translate these numbers to the same thing
> >> > snmpwalk produce, is there any clue on how to achieve that? Is it
> >> > possible at all for different devices (this one happend to be Juniper
> >> > firewall if that does matter). Also, how can I know where does this
> >> > magic oid ends and where does additional information starts (like ip
> >> > addresses added to oid 0 they all looks like another oid string
> >> > numbers) ?
>
> >> > Any sample of code, or hint to another lib will be very appreciated!
>
> >> > --
> >> >
> >> > Spank The Spam!
>
> >> Here's an example of walk that
Re: Translating pysnmp oids to human readable strings
On Mar 5, 1:05 pm, birdsong wrote:
> On Mar 5, 12:32 pm, SpamMePlease PleasePlease
>
>
>
> wrote:
> > Hey list,
>
> > I was given a task, to reproduce functionality of command specified
> > below by writing proper python functions to reuse in some monitoring
> > script:
>
> > rivendell# snmpwalk -Os -mALL -v1 -cgabilgathol 10.0.6.66
> > .1.3.6.1.4.1.2636.5.1.1.2
> > jnxBgpM2PeerIdentifier.0.ipv4."".unknown."".0.1.38.101.87.145 =
> > STRING: 66.250.1.253
> > jnxBgpM2PeerIdentifier.0.ipv4.38.101.161.119.1.38.101.161.118 =
> > STRING: 66.28.1.85
> > jnxBgpM2PeerIdentifier.0.ipv4.64.200.59.74.1.64.200.59.73 = STRING:
> > 64.200.68.12
> > jnxBgpM2PeerIdentifier.0.ipv4.72.37.131.250.1.72.37.131.249 = STRING:
> > 64.235.224.240
> > jnxBgpM2PeerState.0.ipv4."".unknown."".0.1.38.101.87.145 = INTEGER:
> > established(6)
> > jnxBgpM2PeerState.0.ipv4.38.101.161.119.1.38.101.161.118 = INTEGER:
> > established(6)
> > jnxBgpM2PeerState.0.ipv4.64.200.59.74.1.64.200.59.73 = INTEGER:
> > established(6)
> > jnxBgpM2PeerState.0.ipv4.72.37.131.250.1.72.37.131.249 = INTEGER:
> > established(6)
> > (more output)
>
> > I have already found a pysnmp library to fetch the data from the
> > device with a minimal amount of code:
>
> > from pysnmp.entity.rfc3413.oneliner import cmdgen
> > from pysnmp.smi import *
> > import string
>
> > cmdGen = cmdgen.CommandGenerator()
> > errorIndication, errorStatus, errorIndex, varBinds =
> > cmdgen.CommandGenerator().nextCmd(cmdgen.CommunityData('AmonMuil',
> > 'gabilgathol', 0),
> > cmdgen.UdpTransportTarget(('fw-1.datacenter.gondor.net', 161)),
> > (1,3,6,1,4,1,2636,5,1,1,2))
>
> > print errorIndication, errorStatus
> > for i in varBinds:
> > print i
>
> > The problem is that I have completely stuck on the result I am
> > experiencing being totally human unreadable, like this:
>
> > rivendell# python snmp.py
> > None 0
> > [(ObjectName('1.3.6.1.4.1.2636.5.1.1.2.1.1.1.1.0.1.0.0.0.0.1.38.101.87.145'),
> > OctetString('B\xfa\x01\xfd'))]
> > [(ObjectName('1.3.6.1.4.1.2636.5.1.1.2.1.1.1.1.0.1.38.101.161.119.1.38.101.161.118'),
> > OctetString('B\x1c\x01U'))]
> > [(ObjectName('1.3.6.1.4.1.2636.5.1.1.2.1.1.1.1.0.1.64.200.59.74.1.64.200.59.73'),
> > OctetString('@\xc8D\x0c'))]
> > [(ObjectName('1.3.6.1.4.1.2636.5.1.1.2.1.1.1.1.0.1.72.37.131.250.1.72.37.131.249'),
> > OctetString('@\xeb\xe0\xf0'))]
> > [(ObjectName('1.3.6.1.4.1.2636.5.1.1.2.1.1.1.2.0.1.0.0.0.0.1.38.101.87.145'),
> > Integer32('6'))]
> > [(ObjectName('1.3.6.1.4.1.2636.5.1.1.2.1.1.1.2.0.1.38.101.161.119.1.38.101.161.118'),
> > Integer32('6'))]
> > [(ObjectName('1.3.6.1.4.1.2636.5.1.1.2.1.1.1.2.0.1.64.200.59.74.1.64.200.59.73'),
> > Integer32('6'))]
> > [(ObjectName('1.3.6.1.4.1.2636.5.1.1.2.1.1.1.2.0.1.72.37.131.250.1.72.37.131.249'),
> > Integer32('6'))]
> > [(ObjectName('1.3.6.1.4.1.2636.5.1.1.2.1.1.1.3.0.1.0.0.0.0.1.38.101.87.145'),
> > Integer32('2'))]
>
> > Since I cant find any way to translate these numbers to the same thing
> > snmpwalk produce, is there any clue on how to achieve that? Is it
> > possible at all for different devices (this one happend to be Juniper
> > firewall if that does matter). Also, how can I know where does this
> > magic oid ends and where does additional information starts (like ip
> > addresses added to oid 0 they all looks like another oid string
> > numbers) ?
>
> > Any sample of code, or hint to another lib will be very appreciated!
>
> > --
> >
> > Spank The Spam!
>
> Here's an example of walk that's part of a class I wrote, hopefully
> indentation survives the paste. I borrowed heavily from example code
> on the pysnmp site.
>
> from pysnmp import asn1, v2c
> from pysnmp import role
>
> def walk(self, community_string, base_oids):
>
> if type(base_oids) is str: base_oids = [base_oids]
> # this this does what it says, dont bother asking for oids that
> we'll see in our walk
> base_oids = self.remove_child_oids(base_oids)
> # h_pair is just (host, port)
> client = role.manager(self.h_pair)
> client.timeout = 10
> req = v2c.GETNEXTREQUEST(community=community_string)
> rsp = v2c.GETRESPONSE()
> req['encoded_oids'] = map(asn1.OBJECTID().encode, ba
Re: Translating pysnmp oids to human readable strings
On Mar 5, 12:32 pm, SpamMePlease PleasePlease
wrote:
> Hey list,
>
> I was given a task, to reproduce functionality of command specified
> below by writing proper python functions to reuse in some monitoring
> script:
>
> rivendell# snmpwalk -Os -mALL -v1 -cgabilgathol 10.0.6.66
> .1.3.6.1.4.1.2636.5.1.1.2
> jnxBgpM2PeerIdentifier.0.ipv4."".unknown."".0.1.38.101.87.145 =
> STRING: 66.250.1.253
> jnxBgpM2PeerIdentifier.0.ipv4.38.101.161.119.1.38.101.161.118 =
> STRING: 66.28.1.85
> jnxBgpM2PeerIdentifier.0.ipv4.64.200.59.74.1.64.200.59.73 = STRING:
> 64.200.68.12
> jnxBgpM2PeerIdentifier.0.ipv4.72.37.131.250.1.72.37.131.249 = STRING:
> 64.235.224.240
> jnxBgpM2PeerState.0.ipv4."".unknown."".0.1.38.101.87.145 = INTEGER:
> established(6)
> jnxBgpM2PeerState.0.ipv4.38.101.161.119.1.38.101.161.118 = INTEGER:
> established(6)
> jnxBgpM2PeerState.0.ipv4.64.200.59.74.1.64.200.59.73 = INTEGER: established(6)
> jnxBgpM2PeerState.0.ipv4.72.37.131.250.1.72.37.131.249 = INTEGER:
> established(6)
> (more output)
>
> I have already found a pysnmp library to fetch the data from the
> device with a minimal amount of code:
>
> from pysnmp.entity.rfc3413.oneliner import cmdgen
> from pysnmp.smi import *
> import string
>
> cmdGen = cmdgen.CommandGenerator()
> errorIndication, errorStatus, errorIndex, varBinds =
> cmdgen.CommandGenerator().nextCmd(cmdgen.CommunityData('AmonMuil',
> 'gabilgathol', 0),
> cmdgen.UdpTransportTarget(('fw-1.datacenter.gondor.net', 161)),
> (1,3,6,1,4,1,2636,5,1,1,2))
>
> print errorIndication, errorStatus
> for i in varBinds:
> print i
>
> The problem is that I have completely stuck on the result I am
> experiencing being totally human unreadable, like this:
>
> rivendell# python snmp.py
> None 0
> [(ObjectName('1.3.6.1.4.1.2636.5.1.1.2.1.1.1.1.0.1.0.0.0.0.1.38.101.87.145'),
> OctetString('B\xfa\x01\xfd'))]
> [(ObjectName('1.3.6.1.4.1.2636.5.1.1.2.1.1.1.1.0.1.38.101.161.119.1.38.101.161.118'),
> OctetString('B\x1c\x01U'))]
> [(ObjectName('1.3.6.1.4.1.2636.5.1.1.2.1.1.1.1.0.1.64.200.59.74.1.64.200.59.73'),
> OctetString('@\xc8D\x0c'))]
> [(ObjectName('1.3.6.1.4.1.2636.5.1.1.2.1.1.1.1.0.1.72.37.131.250.1.72.37.131.249'),
> OctetString('@\xeb\xe0\xf0'))]
> [(ObjectName('1.3.6.1.4.1.2636.5.1.1.2.1.1.1.2.0.1.0.0.0.0.1.38.101.87.145'),
> Integer32('6'))]
> [(ObjectName('1.3.6.1.4.1.2636.5.1.1.2.1.1.1.2.0.1.38.101.161.119.1.38.101.161.118'),
> Integer32('6'))]
> [(ObjectName('1.3.6.1.4.1.2636.5.1.1.2.1.1.1.2.0.1.64.200.59.74.1.64.200.59.73'),
> Integer32('6'))]
> [(ObjectName('1.3.6.1.4.1.2636.5.1.1.2.1.1.1.2.0.1.72.37.131.250.1.72.37.131.249'),
> Integer32('6'))]
> [(ObjectName('1.3.6.1.4.1.2636.5.1.1.2.1.1.1.3.0.1.0.0.0.0.1.38.101.87.145'),
> Integer32('2'))]
>
> Since I cant find any way to translate these numbers to the same thing
> snmpwalk produce, is there any clue on how to achieve that? Is it
> possible at all for different devices (this one happend to be Juniper
> firewall if that does matter). Also, how can I know where does this
> magic oid ends and where does additional information starts (like ip
> addresses added to oid 0 they all looks like another oid string
> numbers) ?
>
> Any sample of code, or hint to another lib will be very appreciated!
>
> --
>
> Spank The Spam!
Here's an example of walk that's part of a class I wrote, hopefully
indentation survives the paste. I borrowed heavily from example code
on the pysnmp site.
from pysnmp import asn1, v2c
from pysnmp import role
def walk(self, community_string, base_oids):
if type(base_oids) is str: base_oids = [base_oids]
# this this does what it says, dont bother asking for oids that
we'll see in our walk
base_oids = self.remove_child_oids(base_oids)
# h_pair is just (host, port)
client = role.manager(self.h_pair)
client.timeout = 10
req = v2c.GETNEXTREQUEST(community=community_string)
rsp = v2c.GETRESPONSE()
req['encoded_oids'] = map(asn1.OBJECTID().encode, base_oids)
oids_values = {}
while req['encoded_oids']:
try:
answer, host_tuple = client.send_and_receive(req.encode())
except (role.NoResponse, role.NetworkError):
return oids_values
rsp.decode(answer)
parsed_oids_vals = self.parse_response(rsp, base_oids)
oids_values.update(parsed_oids_vals)
req['request_id'] += 1
req['encoded_oids'] = map(asn1.OBJECTID().encode,
parsed_oids_vals.keys())
return oids_values
--
http://mail.python.org/mailman/listinfo/python-list
thread safe to lock on key,val pairs on a dict instead of entire dict?
Dictionaries just store references to objects, right? So is it thread
safe to lock a specific key/val pair on a dictionary and modify its
val and release the lock?
example snippet:
# assuming d_lock was initialized long ago in a thread-safe manner
d_lock.acquire()
d = {}
d[1] = (threading.Lock(), [])
d_lock.release()
# test key level locking
for key, data in d.items():
row_lock, rows = data
row_lock.acquire()
rows.append(1)
row_lock.release()
Of course, I'll have to lock the entire dict when adding keys that
dont exist, but further calls and acquire the key specific lock before
doing any write operations.
--
http://mail.python.org/mailman/listinfo/python-list
Re: single thread decrement a semaphore object more than once?
On Feb 24, 8:57 pm, Christian Heimes wrote: > birdsong wrote: > > I searched but didn't see this already discussed, sorry if I didn't > > search hard enough. > > > Can I decrement a semaphore's counter within the same thread more than > > once? I'd like to set hard and soft limits in a server I'm writing. > > The server upon initialization would create a semaphore: > > > threading.Semaphore(100) > > Yes, you are not restricted to multiple threads. A semaphore ensures > that the counter is thread safe. It doesn't force you to use threads. > > Christian ahh good. there will be multiple threads accessing, but yes i follow now. i guess this would have been pretty easy to test in the shell. muchas gracias. -- http://mail.python.org/mailman/listinfo/python-list
single thread decrement a semaphore object more than once?
I searched but didn't see this already discussed, sorry if I didn't search hard enough. Can I decrement a semaphore's counter within the same thread more than once? I'd like to set hard and soft limits in a server I'm writing. The server upon initialization would create a semaphore: threading.Semaphore(100) Then acquire 50x to hold half of the locks before serving any requests. This allows me to grow the number of locks to the hard limit via signals or some other mechanism, but I need to know that any 1 thread has the ability to acquire up or release more than 1x and effectively acquire and release for that number of semaphore values. I could implement this by using regular threading.Lock in a pool, but the Semaphore object would be much easier. -- http://mail.python.org/mailman/listinfo/python-list
Re: Run a linux system command as a superuser, using a python script
On Feb 24, 11:44 am, Chris Rebert wrote:
> On Tue, Feb 24, 2009 at 11:38 AM, madhav wrote:
> > I have got postfix installed on my machine and I am updating on of its
> > configuration files programmatically(using python)(on some action).
> > Since any change in the configuration needs a reload, I need to reload
> > postfix to reflect the latest change. How can I do that in a python
> > script. Precisely, I have something like this:
>
> > import subprocess
> > subprocess.Popen('sudo /etc/init.d/postifx reload')
>
> > Since this script should be run in no-human environment(on the fly), I
> > cant supply the sudo password or even root password(if needed). Even
> > sudo doesn't have a flag to input the password for the command at the
> > first shot. How do I do this?
>
> pexpect may be useful in this situation -http://www.noah.org/wiki/Pexpect
>
> Cheers,
> Chris
>
> --
> Follow the path of the Iguana...http://rebertia.com
I vote down the pexpect, you'd be hardcoding a password. How about
adjusting the sudoers file to grant the user of this script sudo on /
etc/init.d/postifx or even sudo on your python script.
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python shell scripting and errors
On Feb 23, 9:34 am, Philip Semanchuk wrote:
> On Feb 23, 2009, at 12:21 PM, Tim Wintle wrote:
>
> > On Mon, 2009-02-23 at 09:12 -0800, Phillip B Oldham wrote:
> >> I've got a python script running as a daemon (using someone else's
> >> daemon module). It runs fine for a while, but will occasionally balk
> >> and die. Since its running in the background, I'm getting no error
> >> from it.
>
> >> What's the best way to capture the output into a file for later
> >> review?
>
> > Point standard out to an open file handle?
>
> And stderr.
>
> I don't know how the OP's daemon is being launched, but the crontab
> command below worked for me for redirecting errors to a log file
> without the need to modify the script.
>
> python /usr/local/foo/bar.py >> /var/log/foo/bar.txt 2>&1
If the daemon module is worth it's salt, it will have probably closed
all open filehandles including STDOUT and STDERR. So this might not
work.
If all you want is some temporary debugging, add some redirection for
stdout and stderr after you daemonize.
import sys
sys.stderr = open('/path/to/suitable_stderr_file', 'w')
sys.stdout = open('/path/to/suitable_stdout_file', 'w')
I'm pretty sure this should catch any tracebacks.
>
> HTH
> Philip
--
http://mail.python.org/mailman/listinfo/python-list
Re: select.poll.poll() never blocks
On Feb 11, 10:36 pm, "Hendrik van Rooyen" wrote: > "birdsong" wrote: > > 8<--- select not blocking on empty file stuff - > > > Any help on what I'm missing would be appreciated. > > Why do you expect it to block? > It is ready to read, to return end of file. > I expected it to block because of the name and meaning of the flag, POLLIN, which I thought meant new data has arrived to be read. Since I only registered a single file descriptor, I would have expected poll to not return until there was new data to be read. I understand now that it is not the case. -- http://mail.python.org/mailman/listinfo/python-list
Re: select.poll.poll() never blocks
On Feb 11, 7:47 pm, "Rhodri James" wrote: > On Thu, 12 Feb 2009 03:01:16 -0000, birdsong > wrote: > > > So I guess I didn't have a complete understanding of poll, I thought > > it returned file descriptors that had registered an event that the > > user asked to watch for. In my case, select.POLLIN > > Constant Meaning > > POLLIN There is data to read > > > ...there ins't any data to be read. The file is readable, yes, but is > > that all that the syscall has to offer? > > To quote the web page referenced in help(select): > > "It cannot be used on regular files to determine whether a file has grown > since it was last read." > > poll() and friends are wrappers over the socket library "poll" and > "select" functions, which primarily exist to facilitate asynchronous > socket communications. They can take ordinary file descriptors as > well as sockets because there's no good reason not to, but a regular > file will always be ready to read as long as you haven't read past > the end of file. > > You're trying to use a regular file as if it was a pipe. My first > obvious question is do you have to do it this way? Do you have > any control over the program writing to file? Can you cause the > output to go to stdout and pipe it into your script's stdin > instead? That would make your life vastly easier. agreed, this was my original idea, but i figured implementing a tail - f via poll was alot faster to write and implement. i'm glad at least know why it wasn't working as expected. i've already written a tail -f similar to the one found in the link from a later poster, but i was hoping to use poll to cut down on unecessary seeks -i'm trying to avoid actions that would drive up iowait on the boxes this would run on. thanks for the clarification on the poll family of syscalls. > > If you can't, I'm not sure what your best strategy is. I'd > be tempted to use "tail -f filetocheck | yourscript.py" and > palm the job off on an already-written tool. If you want to > do it in Python, the only thing that springs to mind is > periodically checking the size of the file and reading more > when that changes. You'll need to be very careful to keep > what size you think the file is in sync with how much you've > read! > > -- > Rhodri James *-* Wildebeeste Herder to the Masses -- http://mail.python.org/mailman/listinfo/python-list
Re: select.poll.poll() never blocks
So I guess I didn't have a complete understanding of poll, I thought it returned file descriptors that had registered an event that the user asked to watch for. In my case, select.POLLIN ConstantMeaning POLLIN There is data to read ...there ins't any data to be read. The file is readable, yes, but is that all that the syscall has to offer? On Feb 11, 6:49 pm, Jean-Paul Calderone wrote: > On Wed, 11 Feb 2009 18:44:53 -0800 (PST), birdsong > wrote: > >I'm pretty sure I've exhausted all searches and read all the forums > >Google will turn up related to this issue. > > >I touch an empty file in a sh shell, fire up the python shell, open > >the file for reading(tried all buffering options), register it with a > >poll object for select.POLLIN and call poll(), but the poll never > >blocks and always returns for the FD, EVENT combination I ask for, but > >the file has not anything written to it. > > Filesystem files are always reported as readable and writeable by select, > poll, epoll, etc. > > If you want to do non-blocking filesystem I/O, your choices are to use a > native thread (Python's threading module, or another third-party module > which wraps the platform thread API) or to use POSIX AIO (or the mildly > incompatible Linux variant. Of course, AIO has tons of caveats and is > generally in a miserable state, so you probably shouldn't use it. That > leaves you with threads. > > Jean-Paul -- http://mail.python.org/mailman/listinfo/python-list
select.poll.poll() never blocks
I'm pretty sure I've exhausted all searches and read all the forums
Google will turn up related to this issue.
I touch an empty file in a sh shell, fire up the python shell, open
the file for reading(tried all buffering options), register it with a
poll object for select.POLLIN and call poll(), but the poll never
blocks and always returns for the FD, EVENT combination I ask for, but
the file has not anything written to it.
Here's an example snippet:
>>> fd = os.open('/tmp/poll_test', os.O_RDONLY | os.O_SYNC)
>>> p = select.poll()
>>> p.register(fd, select.POLLIN)
>>> s = p.poll()
>>> os.read(fd, 10)
''
>>> s
[(3, 1)]
>>> fd
3
I was using the open builtin originally, but a forum I read suggested
that this created user level buffers that the system call wouldn't
know about(which didn't completely jive with me), so I switched to the
lower level os.open.
Am I not getting the proper spirit of select.poll? I know that read()
never blocks and I think I read somewhere that poll will return fd
that will not block for the EVENT registered -so in that regard it's
fitting.
Any help on what I'm missing would be appreciated.
--
http://mail.python.org/mailman/listinfo/python-list
Re: "Weird" Indentation? (Or: is there a for...else construct?)
On Feb 7, 6:34 am, Andreas Waldenburger wrote: > On Sun, 08 Feb 2009 01:28:00 +1100 Steven D'Aprano > > wrote: > > Andreas Waldenburger wrote: > > > > It seems that there is a for...else construct. Replacing the inner > > > if with pass seems to confirm this. The else clause is still > > > executed. > > > Yes, there is a for...else construct. > > That's something. In 6+ years of Python programming I've never seen or > heard of this thing. This might be useful, apparently. > yeah, about 4 years for me and i've never noticed this feature. > > [snip] > > > > What's broken here: Python or my brain? > > > Perhaps we should not answer that question. > > I did phrase that rather provocatively, didn't I? > > Well thanks. I'll try to learn less noisily in the future. :) > > /W > > -- > My real email address is constructed by swapping the domain with the > recipient (local part). -- http://mail.python.org/mailman/listinfo/python-list
