Re: Exploiting Dual Core's with Py_NewInterpreter's separated GIL ?
Shane Hathaway wrote: robert wrote: I'd like to use multiple CPU cores for selected time consuming Python computations (incl. numpy/scipy) in a frictionless manner. Interprocess communication is tedious and out of question, so I thought about simply using a more Python interpreter instances (Py_NewInterpreter) with extra GIL in the same process. I expect to be able to directly push around Python Object-Trees between the 2 (or more) interpreters by doing some careful locking. Any hope to come through? If possible, what are the main dangers? Is there an example / guideline around for that task? - using ctypes or so. Or is there even a ready made Python module which makes it easy to setup and deal with extra Interpreter instances? If not, would it be an idea to create such thing in the Python std libs to make Python multi-processor-ready. I guess Python will always have a GIL - otherwise it would loose lots of comfort in threaded programming I'd like to mention mod_python, which creates multiple interpreters inside Apache. It does this transparently and works well. There is no apparent lock contention between the interpreters, because no Python objects are shared. In relation to mod_python, your statement isn't strictly true. One can create scenarios in mod_python where a Python object created in the context of one interpreter is then used in another interpreter at a later point. Generally this isn't an issue because it is part of a recursive sequence of processing and thus while the second interpreter is doing stuff with the object the first is stalled waiting for the second to return. It wouldn't be totally inconceivable though for a user to create threads which may operate on the object at the same time in the first interpreter as the object is being used in the second interpreter. The situation gets even more complicated when you start to use third party modules which may be used from multiple interpreters and a multithreaded Apache MPM is used and where the third party module performs callbacks into interpreters with some common object. Thus, mod_python may not be able to benefit from each Python interpreter instance having its own GIL if that was your point in referencing mod_python. Graham -- http://mail.python.org/mailman/listinfo/python-list
Re: is mod_python borked?
walterbyrd wrote: I am considering python, instead of php, for web-application development. I often see mod_python.criticisized as being borked, broken, or just plain sucking. Any truth to any of that? I replied to you over on the mod_python mailing list when you reposted the question there, but I'll copy to here my response for the benefit of others. In answer to your question of whether mod_python is borked, the answer is yes and no. One sees various people complain about mod_python. In some cases their claims are valid, but in other cases they are wrong and haven't bothered to work out how it works. Even when their claims are valid, they often don't understand the problem properly. To use mod_python properly also means in a lot of cases understanding how Apache works, which many are too lazy to do. For example, the difference between SetHandler and AddHandler and what happens when MultiViews is enabled. One of the biggest problems is that a lot of ISPs still use Apache 1.3 and so only mod_python 2.7.X is available on those platforms. This version is quite old now and doesn't behave exactly the same as more recent versions, yet people read the most recent documentation and complain it doesn't work as they expect. Even where people are using Apache 2.0 and thus can use newer versions of mod_python, they sometimes use RedHat enterprise versions of Linux which are stuck in a time warp with only 3.1.X versions of mod_python available. Some of the other Linux distributions haven't been much better until recently and with people refusing to compile from source code and demanding RPMs, DEBs or other prepackaged releases are only bringing it on themselves. Version 3.2.X has been an improvement over older versions. Significantly it fixed multithreading issues on Windows and with worker MPM on UNIX. For publisher at least, it addressed some of the module importer issues, but not all. It has also addressed some major memory leaks. So in summary, if you use the most recent version you are obviously going to have the least problems. Many still use older versions though and can't or will not upgrade. Even the current version has known issues and you can get a feel for what is known by going through issues at: http://issues.apache.org/jira/browse/MODPYTHON In particular the number of separate issues which will be addressed in 3.3 is getting up towards 100. These aren't all bug fixes, with many being new features and improvements. The most important change will be the replacement of the module importer. For a lengthy description of the problems with the module importer see: http://www.dscpl.com.au/wiki/ModPython/Articles/ModuleImportingIsBroken So, use an older version of mod_python and the answer to the question of whether it is borked is mostly yes. Use the current version and except for module importer issues, the answer is mostly no. With version 3.3, the answer should hopefully be a strong no. Graham -- http://mail.python.org/mailman/listinfo/python-list
Re: PROBLEM with MOD_PYTHON
dan84 wrote: I don't understand this error , in the (Apache) errorlog I read this message : [Sat Oct 28 14:04:03 2006] [error] make_obcallback: could not import mod_python.apache.\n [Sat Oct 28 14:04:03 2006] [error] make_obcallback: Python path being used ['C:Python24python24.zip', '.DLLs', '.lib', '.libplat-win', '.liblib-tk', 'C:ProgrammiApache GroupApache2bin']. [Sat Oct 28 14:04:03 2006] [error] python_handler: no interpreter callback found. [Sat Oct 28 14:04:03 2006] [error] [client 127.0.0.1] python_handler: Can't get/create interpreter. I have Apache/2.0.59 (Win32) mod_python/3.2.10 Python/2.4.3 I have read all the manuals, but I don't understand where I am wrong. thanks so much Marco. Read: http://www.modpython.org/pipermail/mod_python/2006-October/022362.html You may have to fiddle your Windows registry to add site-packages directory. BTW, there is a mailing list specifically for mod_python. You might be better off asking questions directly there, as we have seen most problems. You can also search the mailing list archives from the mod_python site. BTW, the quadruple backslashes is indeed because of the use of repr() when doing the error dump. Graham -- http://mail.python.org/mailman/listinfo/python-list
Re: HTML Templates (Sitemesh/Tiles concept) in Python
Suren wrote: Python with ? CGI ? FastCGI ? mod_python ? Other ? We are using mod_python and SSI. We are inheriting some legacy code that we do not want to mess with at all. If you are already using SSI for basic page composition using 'include virtual', it may be of interest for you to know that upcoming mod_python 3.3 supports use of Python code directly with SSI. For details see: http://www.dscpl.com.au/wiki/ModPython/Articles/BasicsOfServerSideIncludes To use this feature now, you would have to use latest mod_python code base out of Subversion repository though. We are just tidying up last few issues for 3.3, so code is more than usable at this point. There is an example in the article about how the new feature can be used for adding header, footers etc. Graham -- http://mail.python.org/mailman/listinfo/python-list
Re: silent processing with python+modpython+cheetah
Sai Krishna M wrote: Hi, I have been working for some time developing web pages using python, modpython, cheetah. I find that this method has come inherent difficulties in it like if we want to generate a single page we have to write two separate files ( py tmpl). Is there any other better way of doing things.? It sounds like you are actually using mod_python.publisher and hand crafting the code to render the template file in each instance. Using mod_python.publisher is not the same as using basic mod_python handlers directly. If you were using basic mod_python handlers directly, you certainly could write your own dispatcher which encapsulates in one place the code needed to render a Cheetah template file, thereby avoiding the need to create .py file corresponding to every Cheetah template file. Also if we want to have partial processing of the pages, i.e, the value one chooses for a particular field determines the value to be entered for another field in the same page, how can it be done? That sounds like an internal issue of how you use Cheetah templates itself. Cheetah templates have flow control abilities, blocks etc such that one can control what parts of a page are actually used. Since mod_python inherently means you start out working at a low level, and that you don't perhaps fully appreciate the different levels of functionality in mod_python and how to use them, you are possibly better off using a high level framework such as Django or TurboGears where someone else has done all the hard work of making it simple to use. Graham -- http://mail.python.org/mailman/listinfo/python-list
Re: CGI - mod_python
[EMAIL PROTECTED] wrote: Hi, it is a kind of nooby question. Is there a way to transfer a CGI python script to mod_python without rewriting the code? Did you look in the mod_python documentation? http://www.modpython.org/live/current/doc-html/hand-cgi.html It certainly isn't the preferred way of using mod_python. You really should consider porting it properly to either mod_python or as someone else suggested, WSGI. Graham -- http://mail.python.org/mailman/listinfo/python-list
Re: What's the best way to communicate between processes?
[EMAIL PROTECTED] wrote: I'm thinking about coding some sort of 'local notify server' in twisted (basically just listening to a socket and then passing the information to the IM bot), then connecting to it via a python socket in django and passing some serialized/pickled data. Use the XML-RPC server gateway from Twisted on the server side. In the Django web pages then use xmlrpclib module to initiate XML-RPC requests to the server. Graham -- http://mail.python.org/mailman/listinfo/python-list
Re: os.path.normpath
[EMAIL PROTECTED] wrote: I am using a windows box and passing a string like ../foo/../foo2 to normpath which then returns ..\\foo2. But if this string is going into a webpage link it should really be ../foo. Is there any way to tell os.path.normpath to act like we are an a unix style box? Use posixpath.normpath() instead. -- http://mail.python.org/mailman/listinfo/python-list
Re: Event notification system - where to start ?
[EMAIL PROTECTED] wrote: We have been asked to develop and application for a client that is a 'notification system. We would like to use python, but are struggling to find the right starting point. Any suggestions, tips or sample code would be appreciated. Application outline; Machine A is running a listener application that is connected to a another device via the serial post and waits for events. We have not problem working with the serial port, or the waiting for the event to happen. When A received a specific event, it needs to send a message to machine B that and event has occurred and that type of event. Machine B will take the event notification, processes some additional information (ie. database lookups) and then notify a series of clients that have registered with machine B to receive events. I think this is sometimes called published/subscriber ??? This is exactly the sort of thing that OSE is purpose built for. See recipe from Python Cookbook online at: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/81614 http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/81615 OSE can be obtained from: http://ose.sourceforge.net The documentation for the Python wrappers for OSE can be found at: http://ose.sourceforge.net/browse.php?group=python-manualentry=manual.htm Graham -- http://mail.python.org/mailman/listinfo/python-list
Re: [mod_python] using nested blocks in psp
cloc3 wrote: I'm a newbie in python, and I'm fighting against nested blocks in psp. Thats a little example with a funny behaviour: [code] html form select name=List !--makes a list with a loop -- % for i in range(50, 350, 50): if 'List' in form and int(form['List'])==i: sel=selected else: sel=pippo %option value=%=i% %=sel% %=i% /option % % /select input type=submit value=return/ /form /html [/code] In my intention, the for instruction have to run two operation: first: the if instruction second: to print the option tag on the html page. Instead, the real behaviour is depending on the result of the control in the if: if the control is true, the option tag is not printed, and the for instruction goes to the successive step. if the control is false, the option is correctly printed. I don't understand the reason of this behaviour. Which is the right way to control the nested block instructions in psp? PSP as implemented by mod_python is fussy about how many spaces are used in indents. Use either 8 space or tab indents. If you don't want to do that, you will need to use comment hints to help PSP understand what level of indenting you are using. See: http://www.modpython.org/pipermail/mod_python/2005-May/018102.html Comment hints may still be needed in certain cases to turn off scopes even if 8 space or tab indents are needed so it is good to understand what they are about. Also ask further questions on mod_python user mailing list as you will in general get better responses there. Graham -- http://mail.python.org/mailman/listinfo/python-list
Re: [newbie]apache authentication questions
Steve Holden wrote: [EMAIL PROTECTED] wrote: Since HTTP authentication is managed by the browser it's difficult to integrate it with web application authentication: basically you have to choose between the two. There's no way for the server to tell the browser to start presenting the required authentication credentials except by raising a 401 (not authorised) error response, which is what makes the browser bring up its little popup. It is not impossible though and in cases where you don't have a choice but to use a HTTP authentication scheme, use of AJAX may be the answer to still allowing use of a form based login scheme. See: http://www.peej.co.uk/articles/http-auth-with-html-forms.html Graham -- http://mail.python.org/mailman/listinfo/python-list
Re: mod_python web-dav management system
Kyler Laird wrote: Damjan [EMAIL PROTECTED] writes: Now I've been thinking that it might be pretty easy to implement a authorization layer with mod_python (but living handling of the web-dav protocol to apache)... So, has anyone already done something similar? Yup. I'm in the process. I'm using mod_python to do LDAP authentication of WebDAV requests. If original poster needs more in depth help on how to set up an appropriate authhandler for mod_python for the purpose, might suggest they get onto the mod_python mailing list and see help there. Details of mailing list on mod_python web site. Now I just need to get a FUSE filesystem working and I should be able to cobble together a simple DAV server... It would be *so* much cleaner to have a total Python-based solution (that could run as root) so I periodically check on progress in this area. I have high hopes for Twisted's DAV server but I'm told it's not ready for use yet. In another thread I just learned about PanDAV http://ivoras.sharanet.org/projects/pandav.html but I haven't installed it yet. What about if there existed a mod_dav_python for Apache. Would that be of interest. The idea is that the module could bridge the C API hooks and structures of mod_dav module to Python code. This means the benefit from all the hard work that has been put into the Apache mod_dav module and only need to implement stuff that handles the individual operations if you want to implement some sort of custom DAV server. If of interest you might look at comments about it at: http://www.modpython.org/pipermail/mod_python/2006-April/020890.html http://issues.apache.org/jira/browse/MODPYTHON-165 Graham -- http://mail.python.org/mailman/listinfo/python-list
Re: minidom + wxPython woes
Paul Boddie wrote: Frank Millman wrote: Fredrik Lundh wrote: no, it's not a bug in the pyexpat module -- the problem is that wxPython uses it's own incompatible version of the expat library, and loads it in a way that causes problems for any library that's tries to use its own statically linked version. [...] Firstly, it seems from various posts to the tracker item that the same problem has been reported with pygtk, Qt, and VTK. There used to be issues with Expat, PyXML and mod_python which may be vaguely related to this, mostly because there was some usage of Expat within some Apache component which conflicted with PyXML's Expat configuration. FYI, the incompatibility issues that arise with pyexpat in mod_python are well documented at: http://www.dscpl.com.au/articles/modpython-006.html Graham -- http://mail.python.org/mailman/listinfo/python-list
Re: mod_python bytecompiling
Fredrik Lundh wrote: mneagul wrote: I am writing a small program using mod_python publisher extension. I have created several python files and imported them in a `main' file. How can I stop mod_python to byte compile the files? I want to make changes to the files and see result imediately! In the current setup only the changes to the `main' file are applied imediately. I is nearly imposible to develop an application this way. I want to mention that I'm a newby to mod_python. the FAQ has the answer: http://www.modpython.org/FAQ/faqw.py?req=showfile=faq03.001.htp Q. My module imports other modules - how do I reload those? (another approach, used by several frameworks, is to use a stand-alone test server for development, and mod_python for deployment) Simply using apache.import_module() is not going to necessarily help at the moment as it has various problems and shortcomings. One of them is that it does not check multiple levels of dependencies between modules. Thus simply replacing global imports in a module with a call to apache.import_module() will not help. For a full list of problems and issues with the mod_python module importer see: http://www.dscpl.com.au/articles/modpython-003.html There is a new module importer being tested with mod_python 3.3 which will hopefully eliminate the major problems. Highly recommend that original poster get onto the mod_python mailing list to discuss what he can do in the interim. Graham -- http://mail.python.org/mailman/listinfo/python-list
Re: Apache and Python and Ubuntu
[EMAIL PROTECTED] wrote: I've create an Ubuntu Linux box, which comes pre-installed with Python (I've added the libapache2-mod-python throught the app manager). I've created .cgi and .py simple programs in the www root of apache. The problem is the programs just dump the contents to the browser in plain text. Or, in the case of the .py files, I am prompted to download the .py file. How can I get apache to recognize that it should execute the .cgi script? Is there any special thing I need to do to apache to get it to know that the python interpreter should be used for all .cgi files? You do not need mod_python if you want to use Python in traditional CGI scripts. Simply follow the Apache instructions for setting up CGI scripts and ignore mod_python all together. If you really did want to use mod_python, consider reading: http://www.dscpl.com.au/articles/modpython-001.html It provides more detailed instructions on getting a first test handler running. Graham -- http://mail.python.org/mailman/listinfo/python-list
Re: Caching in memory for Apache
Simon Johnson wrote: Dear All, I have decided to take the big plunge and drop the Microsoft platform and use Mod_Python and Apache in it's place. I've never used Linux before this project so it's a really big learning curve, but so far it's going well. I've managed to create some simple pages using the basic of Python. One of the questions I have is: How do you cache stuff in memory? Say I've got a bunch of stuff I need to create from a set of database rows. Obviously, I do not want to consult the database each time I have a request for my page. Is there anyway in Mod_python for me to store values inside memory so I can look it up on the next GET request? It all depends a bit on what you want to cache and whether the data is specific to one users interaction with a page, or whether the data is able to be used across requests from multiple users. This will dictate whether you can use a global data cached within modules used to generate the page, whether you can use sessions, shared memory or whether you have no choice but to always go back to the database. As others have pointed out, don't get trapped in doing premature optimisation. More often than not what you may think will be a bottle neck will not and any problems if they even exist will be elsewhere. In respect of mod_python, it is also vitally important to understand the process/interpreter module under which Python runs within the context of Apache. A starting point for getting information about this is: http://www.dscpl.com.au/articles/modpython-004.html If you do decide that sessions may be useful to what you are doing, as pointed out by others, there can be limitations as to what data can be stored. For limitations in respect of sessions in mod_python have a look at: http://www.dscpl.com.au/articles/modpython-005.html Finally, make sure you use the latest version of mod_python available. Avoid using mod_python 2.7.X or 3.1.X as there are lots of known issues and problems. Not all problems are solved in 3.2.X version, but good progress has been made on getting rid of many issues. The main area which can still cause problems, especially when trying to cache data in the Apache process, is that of module loading and reloading. For a run down on issues in that area see: http://www.dscpl.com.au/articles/modpython-003.html A lot of work has already been done on solving these issues for 3.3.X version of mod_python. BTW, the best place to ask questions about mod_python is the mod_python mailing list. Thus, get yourself subscribed to it. Details of the mailing list are on the mod_python web site. Graham -- http://mail.python.org/mailman/listinfo/python-list
Re: mod_python + publisher + psp + session problems
exhuma.twn wrote: Hi again, as soon as I try to make use of the session object inside a psp-template file, I get the following error: Mod_python error: PythonHandler mod_python.publisher Traceback (most recent call last): File /usr/lib/python2.4/site-packages/mod_python/apache.py, line 299, in HandlerDispatch result = object(req) File /usr/lib/python2.4/site-packages/mod_python/publisher.py, line 143, in handler result = str(result) File /usr/lib/python2.4/site-packages/mod_python/psp.py, line 228, in __str__ self.run() File /usr/lib/python2.4/site-packages/mod_python/psp.py, line 191, in run session = Session.Session(req) File /usr/lib/python2.4/site-packages/mod_python/Session.py, line 389, in Session timeout=timeout, lock=lock) File /usr/lib/python2.4/site-packages/mod_python/Session.py, line 294, in __init__ timeout=timeout, lock=lock) File /usr/lib/python2.4/site-packages/mod_python/Session.py, line 132, in __init__ Cookie.add_cookie(self._req, self.make_cookie()) File /usr/lib/python2.4/site-packages/mod_python/Session.py, line 160, in make_cookie c.path = dirpath[len(docroot):] TypeError: unsubscriptable object Any ideas? Upgrade to mod_python 3.2.8 and your problem will go away. See: http://issues.apache.org/jira/browse/MODPYTHON-50 I suggest that in future you use the mod_python mailing list for questions as that is where most knowledgeable people on mod_python hang out. Graham -- http://mail.python.org/mailman/listinfo/python-list
Re: combining mod_python handlers publisher and psp problem
You could also have done:
def index(req, name='John'):
s = 'Hello, there!'
if name:
names = ['a', 'b', 'c']
s = 'Hello, %s!' % name.capitalize()
tmpl = psp.PSP(req, filename='index.psp')
req.content_type = 'text/html'
tmpl.run(vars = { 'greet': s, 'names': names })
return
Try the mod_python mailing list if you want an explaination of why.
Graham
--
http://mail.python.org/mailman/listinfo/python-list
Re: mod_python and open HTTP connection
Charles wrote: Hello, I'm think about using mod_python for a project but I need to make sure: Does mod_python time out after a minute ? (I hope not). Mod_python itself doesn't time out. If such a thing happened it would be because of how Apache, some intermediate proxy or the client is configured. If I leave an HTTP connection open so that the content keeps loading inside the browser window indefinately, and if I close the browser window, the mod_python process is terminated, right? If the mod_python handler is periodically writing data back to the browser, at the point the socket connection from the client closes, the attempt to write data back to the client by the mod_python handler will result in an IOError exception being raised. Presuming that your handler code doesn't catch this and ignore it, it should cause the handler to abort. Note though that it doesn't cause any process to be terminated as that is not how Apache/mod_python work. It will cause just that current request to be terminated. That thread of control can then be reused to handle a subsequent request. BTW, for tricky questions on mod_python you are better of using the mod_python mailing list. Details on the mod_python web site. Graham -- http://mail.python.org/mailman/listinfo/python-list
Re: apache mod_python problem
Ido Yehieli wrote: Thank you for your response, but I think it's not it - that didn't make any difference. Suggest you read: http://www.dscpl.com.au/articles/modpython-001.html It contains helpful hints for getting a basic handler working in mod_python. If you can get that working, then try something harder. You are also better off asking mod_python questions on the mod_python mailing list. See www.modpython.org home page for how to get on mailing list. Graham -- http://mail.python.org/mailman/listinfo/python-list
Re: Python as a Server vs Running Under Apache
fuzzylollipop wrote: there are lots of things you can't do or can't do easily or can't do at efficiently in Apache using python as cgi or as anyone would more likely assume mod_python. anything that requires any shared state or shared resources in Apache is next to impossible. Doing similar things in an app server or network application framework like twisted is trivial. What one really should be identifying here is that different tools are better for different jobs. Apache is a web server, and it isn't necessarily practical or sensible to try and shoe horn it into service as a full blown application server by and in itself. At the same time though, I would suggest that something like Twisted is in itself not always the best solution either, ie., to take an application server, and try and bolt on to it a web server. Yes Twisted can perform as a web server, but when compared to the functionality and configurability that Apache provides as a web server, it is always going to be a lessor cousin in that role. If you do not need the full power that Apache can provide and can live with the lessor abilities inherent from running the web server as a component within the application server, then great, but this isn't necessarily going to suit everyone either. Thus one is still faced with the question of whether you should embed the application in the web server, or wrap a web server around the application. Unfortunately from what I have seen, often the decision is made based on the narrow focus of expertise of the developers involved. A web developer will be dismissive of the application developer and vice versus. Each will say that it is trivial if done in the system which embodies their area of expertise, although usually that is said without them really knowing how they will achieve the part they don't know too much about. The strongest personalities thus will win as to how the system is built. The alternative is to accept that both Apache and application frameworks such as Twisted are each good in their own space and use both, bridging between the two as necessary. This could be done using simple protocols like XML-RPC or more sophisticated messaging system protocols where persistent connections can exist between the Apache child processes and the backend application server thus avoiding the creation of socket connections on every request. The use of messaging systems also means that presence information can extend into the realm of the web server and it can be easier to deal with failure or unavailability of components of the backed system. Keeping a split like this means you also have the added benefit that you don't end up with application server code which is somehow inherently dependent on or knowing that it is being used purely within the context of a particular web server. You don't end up with application server code being tied to the use of a specific means of rendering web pages. With this sort of split it doesn't matter whether you use CGI, mod_python, mod_perl or PHP for web page generation. Provided that the interfaces between the two are well defined, you can change the application server code structure as much as you like and the web page generation code doesn't need to change. The opposite also applies in as much as changes to the web page generation code shouldn't affect the application server code. Overall I would have to say that you are both right and wrong. Where the state is served purely by the use of the database, there is nothing wrong with Apache/mod_python in itself being used. Where the state also embodies the need to have active agents which operate on that data, possibly independent of web requests, then a back end application server which embeds the core functionality of the application would be better. In doing that though, it doesn't mean that Apache has to be discarded as the web server, and a web server embeded in the application server be adopted. Depending on the requirements, you will get a more flexible, more easily maintained system if a combination of the two technologies is used. Anyway, that statisifies my daily need for a good rant. ;-) Graham -- http://mail.python.org/mailman/listinfo/python-list
Re: Python as a Server vs Running Under Apache
as great as mod_python is, there are lots of restrictions and limitations to what youc an do with it because of limitations of apache itself, and I am refereing to apache 2.x as well as 1.x, like others are saying if you don't need apache specific things it will just be one more thing to work around the design constraints it causes, twisted will be you best bet I don't recollect that anyone had actually mentioned mod_python. Certainly, mod_python isn't the only way of doing things via Apache with Python. Now, I am not trying to defend mod_python, as I probably better than most understand many of the bugs/issues that the current version of mod_python has, but what are the restrictions and limitations in it that you see it has based on it being built on Apache. Anyway, just curious to see what is behind your comment and how deeply you truly understand the inner workings of mod_python, its relationship to Apache and thus its problems. Look forward to you answer. Graham -- http://mail.python.org/mailman/listinfo/python-list
Re: understanding mod_python
PyPK wrote: Hi I'm trying to learn mod python and need some one to explain how to do http redirection(302) with an example code Have you read the documentation? Specifically, check out the function mod_python.util.redirect(): http://www.modpython.org/live/current/doc-html/pyapi-util-funcs.html Mod_python questions are also perhaps better directed at the mod_python users mailing list. Details of mailing list on the mod_python home page: http://www.modpython.org Graham -- http://mail.python.org/mailman/listinfo/python-list
Re: mod_python and threads
Depends on what you mean exactly. If one is on Win32 or using worker mode for Apache, the threads are actually initially created down at the Apache level as part of a common pool and are not bound to a specific Python interpreter. That is, two distinct requests in time could make use of the same thread from the common pool, but may end up calling into the context of two different distinct Python interpreters if it happened that the URL namespace is assigned against different interpreter instances. The data for each Python interpreter is distinct, but if multiple requests come in at the same time which map to the same Python interpreter instance, then multiple threads can be operating within the context of the same Python interpreter instance. If the two threads can be accessing the same code/data, then you would obviously need to worry about threading issues such as locking etc. Anyway, you may have to be clearer about what you want to know. The following may or may not also help. http://www.dscpl.com.au/articles/modpython-004.html BTW, there are bugs related to threading in mod_python 3.1.4 and earlier. Better if you can use mod_python 3.2.5b, which gets rid of the nastier threading issues. Graham -- http://mail.python.org/mailman/listinfo/python-list
Re: ANNOUNCE: Mod_python 3.2.5 Beta
Damjan wrote:
The Apache Software Foundation and The Apache HTTP Server Project are
pleased to announce the 3.2.5 Beta release mod_python.
http://www.modpython.org/live/mod_python-3.2.5b/doc-html/hand-pub-alg-auth.html
says Since functions cannot be assigned attributes,...
But that's not true (at least in 2.3 and 2.4):
def f():
... return 'a'
...
f.__auth__ = {1:'one'}
The documentation was originally written when mod_python supported 2.1
and 2.2. :-)
Anyway, whether it can now be done is irrelevant as the next part of
the documentation states:
Note that this technique will also work if __auth__ or __access__ is
a constant, but will not work is they are a dictionary or a list.
Thus I suspect (but can't confirm right now), that your example will
not work anyway as __auth__ defined within the scope of the function as
a dictionary will not be seen as a constant as required.
Overall, I would not recommend relying on nesting __auth__ or
__access__ within the scope of a published function like this as the
code which supports this feature is somewhat fragile.
http://issues.apache.org/jira/browse/MODPYTHON-43
It is possible that now that mod_python requires at least mod_python
2.3 that new features in the language may be able to be utilised to
improve this feature of mod_python.
BTW, you are better off bringing up issue like this about mod_python on
the mod_python mailing list instead of the general Python group.
Graham
--
http://mail.python.org/mailman/listinfo/python-list
Re: Python/Apache Oddness On OSX
John Abel wrote: Hi, I'm running Python 2.3.5/2.4.2 on OSX 10.4.2, and am trying to run CGI scripts using the builtin Apache. For ease, I've symlinked my custom modules into the /Library/Python/2.3/site-packages directory, and they import OK via command line python. However, when I perform the import from a cgi script, python fails to find the module. It is definately something to do with the symlink, as the CGI works OK if I copy the directory into site-packages. Is there some oddness with Python/Apache and symlink imports? Any pointers would be most useful. If running OS supplied Apache, it runs as the user www. Because this isn't you or root, check that the directory your symlink points at is accessible to others as well as any directories above it back up to the root directory. If it isn't accessible, the user Apache runs as will not be able to find and use the files. When you are copying the directory you are possibly giving it read access for others in the process and that is why it works then. Graham -- http://mail.python.org/mailman/listinfo/python-list
Re: need help with mod_python in RH 9
Read: http://www.dscpl.com.au/projects/vampire/articles/modpython-001.html It addresses the more common problems and how to sort out what you may not have done right. Anyway, I don't seem to see any problems with what you have said you have done, so work through the referenced document. Especially look through the Apache logs as described to ensure that Apache has been restarted and mod_python loaded correctly. Graham -- http://mail.python.org/mailman/listinfo/python-list
Re: need help with mod_python in RH 9
Oh, one minor thing. Your content type should be text/plain and not text/html as you aren't returning HTML This isn't going to be causing the problem you are seeing though. Graham -- http://mail.python.org/mailman/listinfo/python-list
Re: Mod_python psp import module problem
May be related to this bug: http://issues.apache.org/jira/browse/MODPYTHON-12 Where you have: import mod_python.psp change it to: psp = apache.import_module(mod_python.psp) Access the PSP object then as: psp.PSP and not: mod_python.psp.PSP It will happen where somewhere else within the same document tree there is a PythonHandler specified as mod_python.psp as well as the explicit import of mod_python.psp. Depending on which ones gets triggered first, it will screw up. Graham -- http://mail.python.org/mailman/listinfo/python-list
Re: Embedding: many interpreters OR one interpreter with many thread states ?
Greg Ewing wrote: [EMAIL PROTECTED] wrote: - creating many sub-interpreters (Py_NewInterpreter) with a thread state each Or - creating one interpreter with many thread states (PyThreadState_New) My understanding is that using multiple interpeters isn't really supported properly, despite there being apparent support in the API. So I suggest using a single interpeter with multiple threads. Huh. Mod_python uses multiple interpreters and within each interpreter could be running multiple threads at one time corresponding to different incoming HTTP requests. It hasn't helped though that a bug was introduced in Python 2.3.5/4.0 which has been causing grief for some users creating additional threads from within Python code itself. :-( http://sourceforge.net/tracker/index.php?func=detailaid=1163563group_id=5470atid=105470 -- http://mail.python.org/mailman/listinfo/python-list
Re: PSP / mod_python ... how to get POST vars on .psp ?
The documentation states: Additionally, the PSP code will be given global variables req, psp, session and form. In other words, in your PSP page you can access the form variable and it will give you access to the FieldStorage object that PSP creates for you. The members of the FieldStorage class are documented on the mod_python web site. -- http://mail.python.org/mailman/listinfo/python-list
Re: mod_python config problem
David Stanek wrote: On Fri, Jun 03, 2005 at 01:16:17PM -0600, Manuel Pellecer wrote: i want to use mod_python with Apache2 and i made a .htaccess in the subdirectory where i have all my scripts: The .htacces goes like this: AddHandler mod_python .py PythonHandler mptest PythonDebug On Try adding the following line to your .htaccess file: PythonPath sys.path + ['/your/path'] Where '/your/path' is the path in which mptest.py resides. Explicitly overriding PythonPath when using mod_python is not recommended as it causes more problems than good usually. It certainly should not be required to get this working when everything is setup correctly. Suggested that the original poster read: http://www.dscpl.com.au/projects/vampire/articles/modpython-001.html That article is specifically written to help out people with their first mptest example, listing what can go wrong and how to go about working it out. Suggest anything after that is better asked on mod_python mailing list. Graham -- http://mail.python.org/mailman/listinfo/python-list
Re: installing mod_python with python 2.4 on Windows xp
Suggested that you ask this on the mod_python mailing list: http://mailman.modpython.org/mailman/listinfo/mod_python Detail exactly what the problems you are having are. The person who packages the dev versions is on the mod_python list as are other developers of mod_python. I don't know that they always keep up to date on mail on the general Python mailing list. Graham -- http://mail.python.org/mailman/listinfo/python-list
Re: Mod_python
Gensek wrote: Python is obviously binary, but I compiled Apache and modpython. I restarted apache before using the config. I'm inclined to believe that I'm doing everything right, but the computer isn't. Try following through the directions/hints in: http://www.dscpl.com.au/projects/vampire/articles/modpython-001.html A lot of the time people cannot get it working because they use the instructions for mod_python 3.1.X when they actually have 2.7.X installed. The newer instructions will not work on the older version. Anyway, even if you will not follow the steps shown in the article, you can use the hints in it as to what to look for and where. BTW, best place to talk about mod_python is the mod_python mailing list. Graham -- http://mail.python.org/mailman/listinfo/python-list
ANN: Vampire 1.6, an extension module for mod_python.
Vampire 1.6 is now available. http://www.dscpl.com.au/projects/vampire http://www.dscpl.com.au/downloads/vampire-1.6-20050422.tar.gz Vampire is an extension module for mod_python, which provides a more flexible dispatch mechanism for basic content handlers, as well as an alternative implementation of the mod_python.publisher module. A range of other useful features are also provided which make using mod_python a much more pleasant experience. For a quick overview of the features that Vampire provides check out: http://www.dscpl.com.au/projects/vampire/articles/vampire-001.html For a full list of changes in this new version check out: http://www.dscpl.com.au/projects/vampire/changes.html A couple of bugs have been fixed in this version, along with some new features and improvements. The supplied example of how to use mod_python/Vampire with Cheetah templates has also been improved. Problems fixed include a bug in the module import mechanism which was causing thread lockups when import modules were doing sub imports and the system was heavily loaded with requests against the same resource. Plus changes to allow Digest or other forms of authentication to be carried out by Apache and not have Vampire get in the way by throwing back a bad request error due to not knowing anything about that specific authentication type. Most significant new feature is a way of overriding login authentication and access hooks so that an alternate mechanism to Basic authentication can be used. This allows for example a session based mechanism using a web form for login to be easily used. Enjoy. -- http://mail.python.org/mailman/listinfo/python-announce-list Support the Python Software Foundation: http://www.python.org/psf/donations.html
Re: mod_python and zope
David Bear wrote: I will be running zope, and I would also like to run mod_python. The problem arised when zope wants a threaded version of python and mod_python wants no_threads. Threads work fine with mod_python. What makes you think they don't? What versions of Apache/Python/mod_python are you attempting to use? Although threads work okay with mod_python, there are a couple of patches you might want to look at applying to mod_python if you intend using a multithreaded MPM. Details of these can be found at: http://www.dscpl.com.au/projects/vampire/patches.html BTW, for discussion of mod_python issues, the mod_python mailing list might be a better place. http://mailman.modpython.org/mailman/listinfo/mod_python Graham -- http://mail.python.org/mailman/listinfo/python-list
Re: mod_python, user missing
Good question, according to the documentation it should work, I'll push this onto the mod_python mailing list for discussion and get a bug report posted if necessary. In the meantime, you could use the following work around: def __auth__(req, user, passwd): req.user = user if user == 'noppa' and passwd == 'potti': return True else: return False Ie., explicitly set req.user in the __auth__ method. -- http://mail.python.org/mailman/listinfo/python-list
Re: mod_python, user missing
Okay, reason it doesn't work is that req.get_basic_auth_pw() only applies when using Apache itself to perform the user authentication. Ie., where in Apache configuration files you have something like: AuthType Basic AuthName VIP AuthUserFile /tmp/pwdb Require user noppa It doesn't work when using mod_python.publisher style authentication as that is done in mod_python and not by Apache. Thus, you would need to assign to req.user in the __auth__ method as I described to have it so it is accessible in the published method. You will not need to call the req.get_basic_auth_pw() function. -- http://mail.python.org/mailman/listinfo/python-list
Re: mod_python please recompile it with -DEAPI apache warning
[EMAIL PROTECTED] wrote: 2) added to MakeFile after config: EXTRA_CFLAGS=-DEAPI You can do this, but it should be to src/Makefile and added to CFLAGS and not EXTRA_CFLAGS as the latter doesn't exist. -- http://mail.python.org/mailman/listinfo/python-list
Re: python/linux guru needed.. now!!!!
bruce wrote: hi... i have a situation with a linux rh8 server. i can't seem to get python/mod_python/apache working as one... i can't seem to import mod_python from the python interpreter to work, and i'm not sure as to why. i'm fairly convinced that it's a conflict issue of some type, but i'm not sure as to how to resolve it... if you are a guru with python/mod_python/linux then i'd like to talk with you... searching through google/mailing lists/etc... is getting me nowhere!! You will do better if you present your problem in the mod_python mailing list. Go to http://www.modpython.org to get the mailing list details, get yourself on the list and then post the following information as a start. Which version of mod_python you are using. Whether you are using the global configuration file or a .htaccess file. The snippet of the Apache configuration you are using to configure mod_python and enable its use in the directory you are working in. Details of any error responses which are being displayed by your web browser. Details of messages, be they notices or error messages, which occur in the Apache error log file corresponding to any request you are making which is not working. The full contents of the Python code file for the content handler you are trying to use. -- http://mail.python.org/mailman/listinfo/python-list
Re: Moving to Python from PHP - 3 questions
Michal Migurski wrote: 3) Structured request variables. PHP has a really handy feature where request variables with name like foo[], foo[bar], or foo[bar][baz] are automatically structured into nested associative arrays. I can see that the python cgi module will make a list of MiniFieldStorage objects when more than one variable with the same name Someone already piped in by suggesting Vampire for mod_python, so would add that Vampire supports structured form variables by making use of: http://svn.colorstudy.com/trunk/Validator/validator/variabledecode.py Thus, if you follow the form naming conventions this code supports, then data can be turned into lists and dictionaries as appropriate. The only thing is the naming convention is different to what you list above. Read the comments in linked code to see what it does actually use. -- http://mail.python.org/mailman/listinfo/python-list
Re: Moving to Python from PHP - 3 questions
If you do manage to get mod_python working, I suggest taking a look at Vampire as well: http://www.dscpl.com.au/projects/vampire/ I have had good experience with it. Once you start using mod_python you'll realize you can really go anywhere you want; and that's not necessarily a good thing. Vampire points you in a nice direction (I don't want to say 'the right' direction). Vampire is still in a growing phase, so would be interested to hear any comments you may have about how it can be improved. Up until now I have mainly been focusing on making the basic features of mod_python just that bit easier to use. And yes I know that a lot more documentation would be a good start. I am so snowed under with work in my real job at the moment that progress is slow in that respect. :-( -- http://mail.python.org/mailman/listinfo/python-list
Re: Who should security issues be reported to?
Aahz wrote: In article [EMAIL PROTECTED], [EMAIL PROTECTED] wrote: Who are the appropriate people to report security problems to in respect of a module included with the Python distribution? I don't feel it appropriate to be reporting it on general mailing lists. There is no generally appropriate non-public mechanism for reporting security issues. If you really think this needs to be handled privately, do some research to find out which core developer is most likely to be familiar with it. Even before you do that, check SourceForge to find out whether anyone else has reported it as a bug. I find this response a bit dissappointing frankly. Open Source people make such a big deal about having lots of people being able to look at source code and from that discover security problems, thus making it somehow making it better than proprietary source code. From what I can see, if an Open Source project is quite large with lots of people involved, it makes it very hard to try and identify who you should report something to when there is no clearly identifiable single point of contact for security related issues. Why should I have to go through hoops to try and track down who is appropriate to send it to? All you need is a single advertised email address for security issues which is forwarded onto a small group of developers who can then evaluate the issue and forward it on to the appropriate person. Such developers could probably do such evaluation in minutes, yet I have to spend a lot longer trying to research who to send it to and then potentially wait days for some obscure person mentioned in the source code who has not touched it in years to respond, if at all. Meanwhile you have a potentially severe security hole sitting there wating for someone to expliot, with the only saving grace being the low relative numbers of users who may be using it in the insecure manner and that it would be hard to identify the actual web sites which suffer the problem. I'm sorry, but this isn't really good enough. If Open Source wants to say that they are better than these proprietary companies, they need to deal with these sorts of things more professionally and establish decent channels of communications for dealing with it. And yes I have tried mailing the only people mentioned in the module in question and am still waiting for a response. -- http://mail.python.org/mailman/listinfo/python-list
Re: Who should security issues be reported to?
OP: Did you discover this supposed security hole from black-box observation of behavior or by being one of the 'lots of people being able to look at source code', thereby giving evidence to the point? The technique used which is the source of the problem was actually first discovered in a separate package to the Python distribution, but it was known that the same technique was used in a module within the Python distribution. It is quite possible that other third party packages might use it as well, although a few of the main stream packages have been checked and they don't use exactly the same technique so are okay. I could have just ignored the fact that the Python distribution had the problem and worried about the other package only. a) The OP has provided no info about his/her claim. Since the problem at least affects two packages and because of the potential for mischief, I am hardly about to identify the packages concerned, nor describe anything that is going to allow people to work out what the issue is. b) The OP's original post is a classical troll: blast volunteer developers for not having anticipated and planned for a novel situation; argue against things not said, at least now here, not recently; imply that volunteers own him something. Most people with the expertise to detect a security hole would know better. And the reaction is what I have more and more been seeing in Open Source circles. That, is either treat posters like ignoreant newbies who know no better, or assume they are a trolls trying to discredit Open Source. Quite sad really, one tries to do the right thing and gets abused for it. It doesn't matter if a large project may be perceived as being mostly immune to security problems, one could still occur and if it isn't simple to raise such an issue I am sure than some people wouldn't even bother. c) The noise generated because of b) has alerted any malware writers monitering c.l.p for hints about exploitable security holes that there might be one in one of the few modules where such could reasonably be. With approx 200+ modules in the Python distribution I can hardly see how this helps. If I had done what you had wanted in (a) and gave actual information about the problem I would have been narrowing down the problem to less than a dozen modules. You can't have it both ways. OP: If my doubts are wrong and you really do have something to quietly report to the 'authority', then do so, and quit making a noise about it. And so it is was and knowledgeable people are looking at the issue. It should not though have necessitated me making a noise in order to find someone to deal with it in a timely manner. When a proprietary company doesn't have an easy way of reporting problems or seems not to care too much, Open Source people are on top of them like wolves. Why can't Open Source people hold themselves to the same standard. Not sure why I have even bothered to respond to you as it is probably just the sort of attention you want. You even appear to have some history of taking issue with people, even though in one of your own posts you state: Responding to trollish postings. (Jan 26) My personal strategy is to read only as much of trollish threads as I find interesting or somehow instructive, almost never respond, and then ignore the rest. I also mostly ignore discussions about such threads. Terry J. Reedy Maybe you should simply have not responded. Lets see if you now ignore the followup discussion. :-) -- http://mail.python.org/mailman/listinfo/python-list
Who should security issues be reported to?
Who are the appropriate people to report security problems to in respect of a module included with the Python distribution? I don't feel it appropriate to be reporting it on general mailing lists. -- http://mail.python.org/mailman/listinfo/python-list
Re: a ConfigParser wtf moment
True, wasn't thinking. This will affect get() as well. My problem was a slightly different problem. In your case you would have got what you wanted if get()/items() instead of being implemented as: .try: .value = d[option] .except KeyError: .raise NoOptionError(option, section) Was implemented as: .try: .value = self._sections[section][option] .except KeyError: .raise NoOptionError(option, section) That is, get the raw value from the original dictionary for the section. That way you avoid picking up a value from the user supplied vars. It does also avoid picking a key which has come through from the default section, but that could easily be accomodated if that was perceived to be expected behaviour by having the except clause fall through to looking in the default section. Similarly if seen that getting it from vars is okay as well, fall back onto that as file step. All depends on what the perceived overridding priority is. At the moment vars overrides section which overrides default and the document sort of does say that that is what one should expect for vars at least: . Additional substitutions may be provided using the . `vars' argument, which must be a dictionary whose contents overrides . any pre-existing defaults. Probably not what I would have expected either. I would have expected vars to be available for substitution but not for option selection in the first place. It is too late on a Friday, so I may be hallucinating now. :-) -- http://mail.python.org/mailman/listinfo/python-list
Re: Contributor's List
Anand wrote: A list of contributors to Python Cookbook (Second Edition) is available at the following links. Original list courtesy Alex Martelli. Since the book is not yet in print, the lists are still tentative because of potential last minute editing changes. List of first authors o http://harvestman.freezope.org/cookbook/credau.html List of all authors o http://harvestman.freezope.org/cookbook/creds.html Is this mean't to only cover additional entries which were added in the 2nd edition, or is it also mean't to encompass entries which were carried over from the 1st edition as well. If it is both, then the editing must have been quite cut throat as I dropped from 3 entries in the 1st edition to 0 in the 2nd edition. I can sort of understand if the intent was to get rid of entries which referenced packages which weren't regarded as mainstream. I guess it is only 14 years of work down the drain. ;-( -- http://mail.python.org/mailman/listinfo/python-list
Re: Contributor's List
Anand wrote: A list of contributors to Python Cookbook (Second Edition) is available at the following links. Original list courtesy Alex Martelli. Since the book is not yet in print, the lists are still tentative because of potential last minute editing changes. List of first authors o http://harvestman.freezope.org/cookbook/credau.html List of all authors o http://harvestman.freezope.org/cookbook/creds.html Is this mean't to only cover additional entries which were added in the 2nd edition, or is it also mean't to encompass entries which were carried over from the 1st edition as well. If it is both, then the editing must have been quite cut throat as I dropped from 3 entries in the 1st edition to 0 in the 2nd edition. I can sort of understand if the intent was to get rid of entries which referenced packages which weren't regarded as mainstream. I guess it is only 14 years of work down the drain. ;-( -- http://mail.python.org/mailman/listinfo/python-list
Re: modpython, apache and windows
Bob Van Zant wrote: Fortunately most of the Python-for-the-web implementations do not follow closely to the PHP paradigm of web-based programming. There are some, like PSP, that more closely model what PHP does. It is not uncommon to have something like index.py which does hardly anything except fire up a framework that takes care of parsing the rest of the URI and passing control over to the proper script. Using some relatively cryptic features of Apache you can hide the fact that everything goes through index.py (mod_rewrite). I have never really liked how all these different mod_python extensions insist on using their own special file extensions, eg., .psp, .mps etc. All it does is to expose what you are using to implement a site and makes it hard for you to convert to a different implementation mechanism as all your URLs then need to change. As you point out, the only way around it is to use mod_rewrite rules. Overall, from what I have seen all it does is make configuration files harder to understand and spawn lots of newbie questions as to why they can't get anything to work. To me it is more sensible to use REST principles and not use file type extensions at all, or use an extension which reflects the type of content being delivered up and not the mechanism used to generate it. FWIW, the Vampire package tries to address this problem in mod_python by providing a simple mechanism as an extension to mod_python which gives one greater control over the file extension used in a URL and what handler that then maps to. To set it up, one has a single directive in your httpd.conf or htaccess file of the form: PythonHandler vampire You can then place in your directory any number of content handler .py files. Eg. you might have index.py, about.py, report.py etc. In the first instance, no matter what the file type extension on the URL, if the basename in the URL request matches the basename of the content handler file, then it is that file which is the target of the request. Thus, if you used the following URLs, they would map as shown: index.html -- index.py about.html -- about.py report.pdf -- report.py report.csv -- report.py feedback -- feedback.py Now normally the content handler would be called handler(). Using vampire though, you extend the name with the file type, thus: index.html -- will call handler_html() in index.py about.html -- will call handler_html() in about.py report.pdf -- will call handler_pdf() in report.py report.csv -- will call handler_csv() in report.py feedback -- will call handler() in feedback.py Thus, you can use a more sensible file type in the URL without having to resort to mod_rewrite rules. Further, where a specific resource can be represented in multiple formats which are both generated on the fly, you can have a handler for each in the same content handler .py file, eg., as in .pdf and .csv case. You can also follow REST princinples and have no extension at all. You aren't restricted to just having content handlers in a directory though. You can still have other files such .html and .jpg files. If Vampire determines that there is no content handler corresponding to a request, it will decline to service it and pass it back to Apache which will then serve up the raw .html or .jpg file directly. I should point out that Vampire is not a framework in the sense that it doesn't dictate how your pages are rendered. That handler function which is called is basically the same as a stock standard mod_python handler method. Inside that function you still need to provide the code which creates the response, although there is no reason why you can't on a per resource case trigger off a different system such as PSP, mpservlets or HTMLTemplate to generate the HTML. More could be said, but best to check out: http://www.dscpl.com.au/projects/vampire -- Graham Dumpleton -- http://mail.python.org/mailman/listinfo/python-list
