Re: multiprocessing speedup

2015-09-29 Thread Rita
Thanks for the responses. I will create another thread to supply a more realistic example. On Tue, Sep 29, 2015 at 10:12 AM, Oscar Benjamin <oscar.j.benja...@gmail.com > wrote: > On Tue, 29 Sep 2015 at 02:22 Rita <rmorgan...@gmail.com> wrote: > >> I am u

multiprocessing speedup

2015-09-28 Thread Rita
I am using the multiprocessing with apply_async to do some work. Each task takes a few seconds but I have several thousand tasks. I was wondering if there is a more efficient method and especially when I plan to operate on a large memory arrays (numpy) Here is what I have now import

Re: general module auditing

2014-07-03 Thread Rita
On Thu, Jul 3, 2014 at 2:54 AM, Mark Lawrence breamore...@yahoo.co.uk wrote: On 03/07/2014 02:17, Rita wrote: On Wed, Jul 2, 2014 at 2:46 PM, Irmen de Jong irmen.nos...@xs4all.nl mailto:irmen.nos...@xs4all.nl wrote: On 2-7-2014 4:04, Rita wrote: yes, this helps. But I want

Re: general module auditing

2014-07-03 Thread Rita
On Thu, Jul 3, 2014 at 8:36 AM, Mark Lawrence breamore...@yahoo.co.uk wrote: On 03/07/2014 10:27, Rita wrote: On Thu, Jul 3, 2014 at 2:54 AM, Mark Lawrence breamore...@yahoo.co.uk mailto:breamore...@yahoo.co.uk wrote: On 03/07/2014 02:17, Rita wrote: On Wed, Jul 2, 2014

Re: general module auditing

2014-07-02 Thread Rita
On Wed, Jul 2, 2014 at 2:46 PM, Irmen de Jong irmen.nos...@xs4all.nl wrote: On 2-7-2014 4:04, Rita wrote: yes, this helps. But I want to know who uses the module, serpent. So, when I upgrade it or remove it they won't be affected adversely. (Please don't top-post, it makes the discussion

general module auditing

2014-07-01 Thread Rita
i work in a group of developers (15 or so) who are located globally. I would like to know what modules everyone is uses if I ever have to upgrade my python. Is there mechanism which will let me see who is using what? ie, tom,matplotlib bob, pylab nancy, numpy nancy, matplotlib etc... -- ---

Re: general module auditing

2014-07-01 Thread Rita
yes, this helps. But I want to know who uses the module, serpent. So, when I upgrade it or remove it they won't be affected adversely. On Tue, Jul 1, 2014 at 2:16 PM, Irmen de Jong irmen.nos...@xs4all.nl wrote: On 1-7-2014 12:38, Rita wrote: i work in a group of developers (15 or so) who

source code control and documentation

2014-03-02 Thread Rita
Hi, I am trying to achieve this, every time I commit to svn I automatically run hooks to test my libraries, ATM I am doing this manually cd libs PYTHONPATH=. python test_lib.py and if everything passes I do a svn commit -m 'committing code' I don't have access to my svn server so I can't

Re: inheriting a large python code base

2014-02-16 Thread Rita
, 2014 4:45:04 AM UTC+5:30, Roy Smith wrote: In article Cameron Simpson wrote: On 15Feb2014 12:10, Rita wrote: i just inherited a large python code base and I would like to optimize the code (run faster). The application is a scientific application so I really don't understand

inheriting a large python code base

2014-02-15 Thread Rita
hi all, i just inherited a large python code base and I would like to optimize the code (run faster). The application is a scientific application so I really don't understand the internal logic. Here is what I have done so far, profile (cProfile) and was able to see where the code was taking the

Re: mapping objects

2014-02-02 Thread Rita
Thanks for the response Cameron. No amount of 'googling' could provide me with that caliber response :-) So, it seems regardless I would need a database. On Sun, Feb 2, 2014 at 1:58 AM, Cameron Simpson c...@zip.com.au wrote: On 01Feb2014 20:46, Rita rmorgan...@gmail.com wrote: I want

mapping objects

2014-02-01 Thread Rita
Hello, I want to learn more about ORMs so I stumbled upon, SqlAlchemy. If i had a JSON document (or XML, CSV, etc.._) is it possible to convert it to a SQLAlchemy objects? I like the ability to query/filter ( http://docs.sqlalchemy.org/en/rel_0_9/orm/tutorial.html#common-filter-operators) the

Re: data validation when creating an object

2014-01-16 Thread Rita
Thanks everyone for the replies. On Thu, Jan 16, 2014 at 1:36 AM, Cameron Simpson c...@zip.com.au wrote: On 16Jan2014 15:53, Ben Finney ben+pyt...@benfinney.id.au wrote: Roy Smith r...@panix.com writes: Ben Finney ben+pyt...@benfinney.id.au wrote: Who says it's frowned on to do

data validation when creating an object

2014-01-15 Thread Rita
I would like to do some data validation when its going to a class. class Foo(object): def __init__(self): pass I know its frowned upon to do work in the __init__() method and only declarations should be there. So, should i create a function called validateData(self) inside foo? I would

Re: data validation when creating an object

2014-01-15 Thread Rita
Unfortunately, I couldn't find the reference but I know I read it somewhere. Even with a selective search I wasn't able to find it. I think I read it in context of module/class test case writing. I will keep your responses in mind therefore I will put logic in __init__ for data validation.

general ConfigParser question

2013-11-25 Thread Rita
Hi, I was wondering if the default ConfigParser can handle multi line strings (especially in the relate section) For example, if i have system.ini [Global] memory = 1024 [Process A] command = sleep arguments = 100 [Process B] command = nslookup arguments = hostA output = data [Process C]

python process accounting

2013-04-30 Thread Rita
Hi, I was wondering if it possible to write a python wrapper which will account my processes. I would like to account for all the children processes (fork) by looking at their /proc/pid info. Such as memory, io, open files, stats. So, instead of me running /bin/sleep 10, i would like to run it

Re: python process accounting

2013-04-30 Thread Rita
PM, Rita wrote: Hi, I was wondering if it possible to write a python wrapper which will account my processes. I would like to account for all the children processes (fork) by looking at their /proc/pid info. Such as memory, io, open files, stats. So, instead of me running /bin/sleep 10

Re: Python scheduler

2013-02-21 Thread Rita
at 8:04 PM, Rita rmorgan...@gmail.com wrote: Hello, Here is what I am trying to do. (Currently, I am doing this in cron but i need much more granularity). I am trying to run program every 20 secs and loop forever. I have several of these types of processes, some should run every 5 mins

Re: Python scheduler

2013-02-21 Thread Rita
Thanks. This is what I was looking for! On Thu, Feb 21, 2013 at 9:48 AM, Adam Tauno Williams awill...@whitemice.org wrote: On Wed, 2013-02-20 at 23:04 -0500, Rita wrote: Here is what I am trying to do. (Currently, I am doing this in cron but i need much more granularity). I am trying

children process

2013-02-08 Thread Rita
hi, I currently have a bash wrapper which executes a program, something like this #!/usr/bin/env bash export LD_LIBRARY_PATH=/foo:$LD_LIBRARY_PATH exec $@ I would like to have a python process which will do process accounting for all children, so if a process starts, I would like to get all the

building python from source

2013-01-03 Thread Rita
For those building python from source what are some tests you do to make sure the compilation and installation is up to standard. For instance here are some thing I do: Tk functionality sqlite module Python is compiled with shared object (important for wsgi) Proper preloading of python libraries

Re: ANN: Urwid 1.1.0 - Usability and Documentation

2012-10-28 Thread Rita
+1 great library. I suppose it will never be in the standard library? :p On Wed, Oct 24, 2012 at 5:08 AM, Jean-Michel Pichavant jeanmic...@sequans.com wrote: - Original Message - Announcing Urwid 1.1.0 -- Urwid home page: http://excess.org/urwid/

unittest for system testing

2012-10-17 Thread Rita
Hi, Currently, I use a shell script to test how my system behaves before I deploy an application. For instance, I check if fileA, fileB, and fileC exist and if they do I go and start up my application. This works great BUT I would like to use python and in particular unittest module to test my

Re: unittest for system testing

2012-10-17 Thread Rita
thanks. I suppose I would need a simple example from one of these libraries. ( i typed too soon for , no code needed ) On Wed, Oct 17, 2012 at 8:49 PM, Mark Lawrence breamore...@yahoo.co.ukwrote: On 18/10/2012 01:22, Rita wrote: Hi, Currently, I use a shell script to test how my system

Re: best way to handle this in Python

2012-07-20 Thread Rita
...@ix.netcom.comwrote: {NOTE: preferences for comp.lang.python are to follow the RFC on netiquette -- that is, post comments /under/ quoted material, trimming what is not relevant... I've restructured this reply to match} On Thu, 19 Jul 2012 21:28:12 -0400, Rita rmorgan...@gmail.com declaimed the following

best way to handle this in Python

2012-07-19 Thread Rita
Hello, I have data in many files (/data/year/month/day/) which are named like YearMonthDayHourMinute.gz. I would like to build a data structure which can easily handle querying the data. So for example, if I want to query data from 3 weeks ago till today, i can do it rather quickly. each

Re: best way to handle this in Python

2012-07-19 Thread Rita
Using linux 2.6.31; Python 2.7.3. I am not necessary looking for code just a pythonic way of doing it. Eventually, I would like to graph the data using matplotlib On Thu, Jul 19, 2012 at 8:52 PM, Dave Angel d...@davea.name wrote: On 07/19/2012 07:51 PM, Rita wrote: Hello, I have data

recompiling standard modules

2012-07-09 Thread Rita
I noticed that active state python Tk inter isnt compiled with --enable-threads therefore I would like to recompile the module with a new version of TCL/TK which is compiled with threads. How can I do this? -- --- Get your facts first, then you can distort them as you please.-- --

other languages API to python

2012-05-24 Thread Rita
Hello, A vendor provided a C, C++ and Java API for a application. They dont support python so I would like to create a library for it. My question is, how hard/easy would it be to create something like this? Is there a simple HOWTO or examples I can follow? Can someone shed home light on this?

standard module xpath

2012-05-18 Thread Rita
Hello, Does python ship with xpath query language support? Certainly, I can use lxml but I prefer something in the standard modules list. -- --- Get your facts first, then you can distort them as you please.-- -- http://mail.python.org/mailman/listinfo/python-list

bash/shell to python

2012-05-16 Thread Rita
Hello, I currently build a lot of interfaces/wrappers to other applications using bash/shell. One short coming for it is it lacks a good method to handle arguments so I switched to python a while ago to use 'argparse' module. Its a great complement to subprocess module. I was wondering if there

compiling Tkinter

2012-05-15 Thread Rita
Hello, I understand Tkinter is part of the python distribution but for me it always fails when I try to load the module. I get: import Tkinter ... import _tkinter # if this fails your Python may not be configured for Tk ImportError: No module named _tkinter So, here is how I am compiling

python library to generate restructured text

2012-03-03 Thread Rita
Hello, I am using sqlite3 modules to get data out of a table. I would like to dump the table into a restructured-text (docutil) format. I was wondering if there was a library which would accomplish this. -- --- Get your facts first, then you can distort them as you please.-- --

Re: standalone python web server

2012-02-09 Thread Rita
yes, I would like to use a framework. I like the twisted method the user posted. Are there any examples of it using a framework, get/post, etc..? On Thu, Feb 9, 2012 at 6:28 AM, Thomas Bach thb...@students.uni-mainz.dewrote: Rita rmorgan...@gmail.com writes: I am building a small intranet

standalone python web server

2012-02-08 Thread Rita
I am building a small intranet website and I would like to use Python. I was wondering if there was a easy and medium performance python based web server available. I would like to run it on port :8080 since I wont have root access also I prefer something easy to deploy meaning I would like to

working with a large file

2011-09-12 Thread Rita
I have a large file, 18gb uncompressed, and I would like to know what is the preferred method to read this file for random access. I have several processes reading the file which different calculate arguments. My server has 64gb of memory. Not sure what is the preferred way to do this? -- ---

code generation

2011-08-02 Thread Rita
Hello, This isn't much of a python question but a general algorithm question. I plan to input the following string and I would like to generate something like this. input: a-(b,c)-d output: parent a, child b c parent b c child d Are there any libraries or tools which will help me evaluate

complex data structures in python

2011-05-02 Thread Rita
Hello, I was wondering if anyone has any documentation/recipes for implementing complex data structures. For instance, if you had a dictionary with a list inside a list inside a set. -- --- Get your facts first, then you can distort them as you please.-- --

PyCon

2011-03-11 Thread Rita
Hello, I noticed the old presentations at PyCon aren't avaliable anymore. http://us.pycon.org/2010/ http://us.pycon.org/2009/ Does anyone know where else I can get these presentations? -- --- Get your facts first, then you can distort them as you please.-- --

Re: parsing a file for analysis

2011-02-26 Thread Rita
. And next time I read thru this file, I can skip all of Jan 01, 2011 On Sat, Feb 26, 2011 at 10:29 AM, Andrea Crotti andrea.crott...@gmail.comwrote: Il giorno 26/feb/2011, alle ore 06.45, Rita ha scritto: I have a large text (4GB) which I am parsing. I am reading the file to collect

Re: parsing a file for analysis

2011-02-26 Thread Rita
, Feb 26, 2011 at 10:53 AM, Martin Gregorie martin@address-in-sig.invalid wrote: On Sat, 26 Feb 2011 16:29:54 +0100, Andrea Crotti wrote: Il giorno 26/feb/2011, alle ore 06.45, Rita ha scritto: I have a large text (4GB) which I am parsing. I am reading the file to collect stats

Re: subprocess pipe question

2011-02-25 Thread Rita
The results look right! I did a rather large test and the checksum passed. I will hold off any speed ups as you suggested. On Wed, Feb 23, 2011 at 8:37 PM, Rob Williscroft r...@rtw.me.uk wrote: Rita wrote in news:AANLkTi=88dcpm_kqrs2g620obsnxz0majubfwpeme...@mail.gmail.com

parsing a file for analysis

2011-02-25 Thread Rita
I have a large text (4GB) which I am parsing. I am reading the file to collect stats on certain items. My approach has been simple, for row in open(file): if INFO in row: line=row.split() user=line[0] host=line[1] __time=line[2] ... I was wondering if there is a framework

Re: subprocess pipe question

2011-02-23 Thread Rita
This is a good solution thanks. You should wiki this somewhere. For extra points is there a way to speed up the p.stdout.read(bufsize) ? On Tue, Feb 22, 2011 at 7:57 PM, Rob Williscroft r...@rtw.me.uk wrote: Rita wrote in news:AANLkTi=w95gxosc1tkt2bntgjqys1cbmdnojhokq4...@mail.gmail.com

subprocess pipe question

2011-02-22 Thread Rita
I have a process like this, def run(cmd): #cmd=a process which writes a lot of data. Binary/ASCII data p=subprocess.Popen(cmd,stdout=subprocess.PIPE) I would like to get cmd's return code so I am doing this, def run(cmd): p=subprocess.Popen(cmd,stdout=subprocess.PIPE) rc=p.poll()

Re: subprocess pipe question

2011-02-22 Thread Rita
, 2011 at 3:44 PM, Rita rmorgan...@gmail.com wrote: I have a process like this, def run(cmd): #cmd=a process which writes a lot of data. Binary/ASCII data p=subprocess.Popen(cmd,stdout=subprocess.PIPE) I would like to get cmd's return code so I am doing this, def run(cmd): p

Converting from shell to python

2011-02-21 Thread Rita
Hello, I have been using shell for a long time and I decided to learn python recently. So far I am liking it a lot especially the argparse module which makes my programs more professional like. Currently, I am rewriting my bash scripts to python so I came across a subprocess and environment