Re: Guide for Configuring python(1) with httpd(8)
Thanks for all the responses.
I'll respond to everyone (Omar, Crystal, Stuart, Bodie)
within this message to prevent sending out multiple messages.
It appears my message was significantly less clear than I thought.
My apologies.
Let me try again and
then I'll respond directly to messages further down.
What I'm trying to do is display a python file
that has imbedded HTML within a web browser.
The file is 'hello_world.py' and below are its contents:
#!/usr/local/bin/python3
print('Content-type:text/html\n')
print('')
print('')
print('Hello World')
print('')
print('\n')
Within a browser, I want to be able to access this file
and see 'Hello World' displayed.
Where does this file need to be placed?
What configuration changes need to be made to make this work?
Currently, httpd(8) is running
and I'm able to access HTML files via a web browser.
I've seen mentions online that I needed to run slowcgi(8)
in order to get this to work, so I did 'rcctl enable slowcgi' and
'rcctl start slowcgi'.
Then, I placed 'hello_world.py' in '/var/www/htdocs/test/'
and then added the 'location' configuration to '/etc/httpd.conf'.
Below is the full 'httpd.conf':
server "test" {
listen on 10.1.1.1 port 80
root "/htdocs/test"
location "*.py" {
fastcgi
root "/htdocs/test"
}
}
types {
include "/usr/share/misc/mime.types"
}
The reason I was messing with python(1) in chroot
was because that's what I was seeing online.
If that doesn't need to be messed with, great.
However, I don't know what I'd need to do to make this work.
On Sunday, December 18th, 2022 at 11:18 AM, Omar Polo wrote:
> not exactly, fastcgi is a binary protocol, whereas from what you're
> writing I'm assuming you're trying to run a CGI script written in
> python with slowcgi.
>
> (this is what I meant with "explain what you're trying to do" as there
> is a big difference between running a python web application and
> running a custom CGI script.)
Sorry. Hopefully what I have at the top of this message
makes it more clear.
I'm guessing the answer is I'm trying to run a python web application,
but honestly, I'm not sure.
When you say 'custom CGI script', that just confuses me.
I'm assuming that's more backend web development
where data is passed from the server via CGI (slowcgi/fastcgi?)
to python(1).
Whereas python web application
is more of the front end web development
that controls how the page itself looks.
Currently, I'm just trying to load a python file to display content.
> On pypi there is a 'fastcgi' library.
I'll have to take a look at this.
I didn't think I'd need to install an additional library
if I was just trying to display some HTML content.
On Sunday, December 18th, 2022 at 11:38 AM, Crystal Kolipe
wrote:
> It's probably about one day's work.
What may be one day's worth of work for you may be weeks/months
for me.
I don't forsee this being that easy for me.
On Sunday, December 18th, 2022 at 1:11 PM, Stuart Henderson
wrote:
> Surely the goal is to run some particular software and make it available
> via an httpd(8) frontend and not just run python(1) i.e. the Python
> interpreter?
Actually, I think this is exactly what I'm trying to do right now.
Maybe it's stupid to do this,
but I figured I'd start just with python.
After getting that to work and understanding it,
then maybe use additional software with python.
See my original message at the top that hopefully better explains
what I'm trying to do.
>fastcgi
> is more common with other languages but also possible with Python and
> would often be a better fit than trying to run Python directly in a chroot
> via httpd->fastcgi->slowcgi->python(1).
Any guides, resources, or rough instructions on how to do this?
On Sunday, December 18th, 2022 at 3:21 PM, Bodie wrote:
> As was already pointed out by others it may be good to know what
> are you trying to achieve actually aka why you are trying to put
> python in chroot. Especially as you said it works with perl fine
> and that one is in base.
Apologies. Hopefully my message at the top is more clear now.
Re: Guide for Configuring python(1) with httpd(8)
On 18.12.2022 08:07, indivC wrote: Can anyone provide a guide for this or rough instructions? I'm running httpd(8) and trying to utilize a python(1) script with an html file. I've got this working using perl(1). However, it doesn't work with python(1) when following the same steps. As was already pointed out by others it may be good to know what are you trying to achieve actually aka why you are trying to put python in chroot. Especially as you said it works with perl fine and that one is in base. My python(1) version is 3.9.15 My OpenBSD version is 7.2 stable. First, I use ldd(1) to determine what files I need to copy for python(1). Second, in '/var/www/', I create all the folder paths from the ldd(1) output. Third, I change the ownership of all the folders to be 'www:www'. Then, I copy all the files from the ldd(1) output into their respective folders. Lastly, I change the permissions of all these files to 750. At this point, before doing anything with httpd(8), I try testing to ensure everything is correct with python(1). If I run 'python3 /var/www/htdocs/test/test.py', the script runs. However, if I run 'chroot /var/www htdocs/test/test.py', I get the following error: "ld.so: python3: can't load library 'libintl.so.7.0'" 'libintl.so.7.0' was one of the files that appeared in ldd(1). ldd(1) said the path was '/usr/local/lib/' and I've got the file copied to '/var/www/usr/local/lib/', so I'm unsure why python(1) is saying it can't load it. What's a little strange is if I run the 'chroot' command from above several times in a row, I'll see the same error for 'libpython3.9.so.0.0' as well. ldd(1) also showed this file and I also have it, so I'm not really sure what the problem is. I've seen some posts indicate that I also need to copy '/sbin/ldconfig' to '/var/www/'. Then, run 'chroot /var/www sbin/ldconfig /usr/local/lib/'. When I do that, I get the following error: "ldconfig: /var/run/ld.so.hints.: No such file or directory" In the above error, '' seems to be some pseudorandom value. If I run the command multiple times, that value changes every time. For instance, one value I got is 'IkB2akBOKX'. Maybe the steps to configure this are a little different for python(1) when compared to perl(1). This is why I was hoping someone could provide a link to a guide or provide some general steps for configuring this. Currently, I've just been trying to piece together what I've been able to find online. Thanks.
Re: Guide for Configuring python(1) with httpd(8)
On 2022-12-18, indivC wrote: > On Sunday, December 18th, 2022 at 9:04 AM, Omar Polo > wrote: > >> instead of asking how to do X so that you can do Y, ask directly how >> to do Y. > > I did. The first line of my message was > "Can anyone provide a guide for this or rough instructions?", > which is in reference to the subject > "Guide for Configuring python(1) with httpd(8)". Surely the goal is to run some particular software and make it available via an httpd(8) frontend and not just run python(1) i.e. the Python interpreter? > Did i precede to explain > how I was trying to attempt to accomplish Y with X? > Yes, but I don't see why that would be a problem. > I feel like it's better for users to actually attempt to try > and solve their problems then not to try at all. Yes but when you would like help, describe and ask about the base problem rather than something further down the line of your idea if how to do it. >> Why do you need python at all in the chroot? Installing all the >> needed files (and keeping them up-to-date!) manually in a chroot is a >> pain. > > I completely agree. > Virtually every solution I've seen online does this. Every solution for "run stuff in chroot" does this, but there are other more common ways to expose Python software to web clients. scgi is possibly the most common but openbsd's base httpd(8) doesn't support that. fastcgi is more common with other languages but also possible with Python and would often be a better fit than trying to run Python directly in a chroot via httpd->fastcgi->slowcgi->python(1). >> Since httpd speaks fastcgi, why not write some python code that >> accepts the requests over fastcgi? (assuming this is what you're >> trying to do, but you didn't tell.) > > I believe that is exactly what I'm trying to do. > The end goal is to be able access a python(1) file from httpd(8). > My understanding is you have to configure slowcgi(8), > which utilizes fastcgi, within httpd(8). The process accepting fastcgi requests would often run (or at least be started from) *outside* the chroot. Separate process waiting and listening for requests, not something executed by the httpd process. Another alternative is to use slowcgi but disable the chroot (-p /). Is chroot really buying you much if you provide an environment making it easier to run things inside that chroot anyway? -- Please keep replies on the mailing list.
Re: Guide for Configuring python(1) with httpd(8)
On Sun, Dec 18, 2022 at 12:18:32PM +0100, Omar Polo wrote: > On pypi there is a 'fastcgi' library. it's not packaged on OpenBSD > and I can't asses how good it is Alternatively, just write a fastcgi handler from scratch - the protocol is fairly simple and fully documented. Httpd only implements a minimalist subset of fastcgi functions anyway, so any fastcgi implementation that's designed to work with it only needs to implement the same subset. It's probably about one day's work.
Re: Guide for Configuring python(1) with httpd(8)
On 2022/12/18 10:23:39 +, indivC wrote:
> On Sunday, December 18th, 2022 at 9:04 AM, Omar Polo
> wrote:
> > Since httpd speaks fastcgi, why not write some python code that
> > accepts the requests over fastcgi? (assuming this is what you're
> > trying to do, but you didn't tell.)
>
> I believe that is exactly what I'm trying to do.
> The end goal is to be able access a python(1) file from httpd(8).
> My understanding is you have to configure slowcgi(8),
> which utilizes fastcgi, within httpd(8).
> I've been able to configure that without any problems using perl(1),
> but not with python(1).
not exactly, fastcgi is a binary protocol, whereas from what you're
writing I'm assuming you're trying to run a CGI script written in
python with slowcgi.
(this is what I meant with "explain what you're trying to do" as there
is a big difference between running a python web application and
running a custom CGI script.)
there are less ugly hacks (IMHO) than one can do with slowcgi instead
of installing huge things like scripting languages inside the /var/www
chroot, but...
> With python(1), because the problem is occurring
> before I even get to httpd(8), I left that out of the message.
> If I attempt to access the python(1) file currently,
> I still see the same 'ld.so' error message I mentioned before.
> Therefore, I was trying to leave out httpd(8)
> and slowcgi(8) configuration as that isn't where the problem lies.
>
> As mentioned before, if you have a better solution for this,
> please share.
> Any links you can provide for step by step instructions
> or general steps on how to accomplish this would be helpful.
>
> Thanks.
...I think that instead of installing python in the /var/www chroot
(plus all the dependencies you'd need) if you really want to use httpd
and write your stuff in python you may have your python script talk
fastcgi instead.
On pypi there is a 'fastcgi' library. it's not packaged on OpenBSD
and I can't asses how good it is, I'm not reccomanding it (just did a
random search on the web and was the first result), but at least seems
to work: (modeled after the example code in the github repo)
#!/usr/bin/env python
# hello.py
from fastcgi import *
from socketserver import TCPServer
class MyHandler(FcgiHandler):
def handle(self):
self.print('Content-type: text/plain\n')
self.print("Hello, world!")
addr = ('localhost', )
with TCPServer(addr, MyHandler) as srv:
srv.serve_forever()
and here's the matching httpd.conf bit
server "localhost" {
listen on * port 80
fastcgi socket tcp localhost
}
this has the advantage of not needing to fork one process per request
like CGI would do, but you need to secure your application by yourself
(running it as a different user for starters -- no idea if you can do
fancier things with python.)
Re: Guide for Configuring python(1) with httpd(8)
On Sunday, December 18th, 2022 at 8:38 AM, Mark Willson > This is the script I use to set up python for httpd: I looked over the script and it seems to do pretty much what I've already done. It copies some additional files that I didn't copy. My first attempt was just to copy over the additional files that I had not previously copied that are in your script. This didn't change anything for me. I then decided to run your script directly. I removed the parts in the script that dealt with touching any folder path with 'run'. 'slowcgi.sock' is in '/var/www/run/' and I didn't want to mess with it. Also, it doesn't look like the script does anything with files in these folders, so it shouldn't matter that I omitted them. On the first run, it wasn't able to copy 'libiconv.so.7.0'. On my system, it's 'libiconv.so.7.1'. Therefore, I updated that line in the script to 'libiconv.so.*'. This better matches how all the other lines are in the script. I'm not sure why this line and the one above it are different. Then, I ran the script again. However, I still get the same 'chroot' error: "ldconfig: /var/run/ld.so.hints.: No such file or directory" So I'm not sure why I'm getting different results than you. It's like something is attempting to generate a pseudorandom file using 'ld.so.hints' as a base. These pseudorandom filenames do not exist, but '/var/run/ld.so.hints' does, so I'm not sure why it doesn't just use that file directly.
Re: Guide for Configuring python(1) with httpd(8)
On Sunday, December 18th, 2022 at 9:04 AM, Omar Polo wrote: > instead of asking how to do X so that you can do Y, ask directly how > to do Y. I did. The first line of my message was "Can anyone provide a guide for this or rough instructions?", which is in reference to the subject "Guide for Configuring python(1) with httpd(8)". Did i precede to explain how I was trying to attempt to accomplish Y with X? Yes, but I don't see why that would be a problem. I feel like it's better for users to actually attempt to try and solve their problems then not to try at all. > Why do you need python at all in the chroot? Installing all the > needed files (and keeping them up-to-date!) manually in a chroot is a > pain. I completely agree. Virtually every solution I've seen online does this. If there's a better way, I'm all ears. I think it's reasonable to assume if someone is attempting to do something and you believe you have a much better way of accomplishing it, then share it. I never said this is the only way to do it or that this is the only way I want to accomplish this problem. I just shared what I've been trying to get to work, which is based on the information I've found online. > Since httpd speaks fastcgi, why not write some python code that > accepts the requests over fastcgi? (assuming this is what you're > trying to do, but you didn't tell.) I believe that is exactly what I'm trying to do. The end goal is to be able access a python(1) file from httpd(8). My understanding is you have to configure slowcgi(8), which utilizes fastcgi, within httpd(8). I've been able to configure that without any problems using perl(1), but not with python(1). With python(1), because the problem is occurring before I even get to httpd(8), I left that out of the message. If I attempt to access the python(1) file currently, I still see the same 'ld.so' error message I mentioned before. Therefore, I was trying to leave out httpd(8) and slowcgi(8) configuration as that isn't where the problem lies. As mentioned before, if you have a better solution for this, please share. Any links you can provide for step by step instructions or general steps on how to accomplish this would be helpful. Thanks.
Re: Guide for Configuring python(1) with httpd(8)
instead of asking how to do X so that you can do Y, ask directly how to do Y. Why do you need python at all in the chroot? Installing all the needed files (and keeping them up-to-date!) manually in a chroot is a pain. Since httpd speaks fastcgi, why not write some python code that accepts the requests over fastcgi? (assuming this is what you're trying to do, but you didn't tell.) On 2022/12/18 07:07:20 +, indivC wrote: > Can anyone provide a guide for this or rough instructions? > I'm running httpd(8) and trying to utilize a python(1) script > with an html file. > I've got this working using perl(1). > However, it doesn't work with python(1) when following the same steps. > > My python(1) version is 3.9.15 > My OpenBSD version is 7.2 stable. > > First, I use ldd(1) > to determine what files I need to copy for python(1). > Second, in '/var/www/', > I create all the folder paths from the ldd(1) output. > Third, I change the ownership of all the folders to be 'www:www'. > Then, I copy all the files from the ldd(1) output > into their respective folders. > Lastly, I change the permissions of all these files to 750. > > At this point, before doing anything with httpd(8), > I try testing to ensure everything is correct with python(1). > If I run 'python3 /var/www/htdocs/test/test.py', the script runs. > However, if I run 'chroot /var/www htdocs/test/test.py', > I get the following error: > "ld.so: python3: can't load library 'libintl.so.7.0'" > > 'libintl.so.7.0' was one of the files that appeared in ldd(1). > ldd(1) said the path was '/usr/local/lib/' > and I've got the file copied to '/var/www/usr/local/lib/', > so I'm unsure why python(1) is saying it can't load it. > > What's a little strange > is if I run the 'chroot' command from above several times in a row, > I'll see the same error for 'libpython3.9.so.0.0' as well. > ldd(1) also showed this file and I also have it, > so I'm not really sure what the problem is. > > I've seen some posts indicate > that I also need to copy '/sbin/ldconfig' to '/var/www/'. > Then, run 'chroot /var/www sbin/ldconfig /usr/local/lib/'. > When I do that, I get the following error: > "ldconfig: /var/run/ld.so.hints.: No such file or directory" > > In the above error, '' seems to be some pseudorandom value. > If I run the command multiple times, that value changes every time. > For instance, one value I got is 'IkB2akBOKX'. > > Maybe the steps to configure this are a little different for python(1) > when compared to perl(1). > This is why I was hoping someone could provide a link to a guide > or provide some general steps for configuring this. > Currently, I've just been trying to piece together > what I've been able to find online. > > Thanks.
Re: Guide for Configuring python(1) with httpd(8)
> -Original Message-
> From: owner-m...@openbsd.org On Behalf Of indivC
> Sent: 18 December 2022 07:07
> To: misc@openbsd.org
> Subject: Guide for Configuring python(1) with httpd(8)
>
> Can anyone provide a guide for this or rough instructions?
> I'm running httpd(8) and trying to utilize a python(1) script
> with an html file.
[snipped]
> This is why I was hoping someone could provide a link to a guide
> or provide some general steps for configuring this.
> Currently, I've just been trying to piece together
> what I've been able to find online.
>
> Thanks.
Hi indivC,
This is the script I use to set up python for httpd:
#!/bin/sh
#
# Enable Python3 to operate in a chrooted environment (for httpd)
#
export CHROOT=/var/www
rm -rf ${CHROOT}/usr ${CHROOT}/var ${CHROOT}/etc ${CHROOT}/sbin \
${CHROOT}/run ${CHROOT}/logs
mkdir -p ${CHROOT}/usr/local/bin ${CHROOT}/usr/lib ${CHROOT}/usr/libexec \
${CHROOT}/sbin ${CHROOT}/var/run ${CHROOT}/etc \
${CHROOT}/usr/local/lib ${CHROOT}/run ${CHROOT}/logs
chown www ${CHROOT}/usr/local/bin ${CHROOT}/usr/lib ${CHROOT}/usr/libexec \
${CHROOT}/sbin ${CHROOT}/var/run ${CHROOT}/etc \
${CHROOT}/usr/local/lib ${CHROOT}/run ${CHROOT}/logs
cp -p /sbin/ldconfig ${CHROOT}/sbin
cp -p /usr/local/bin/python3.9 ${CHROOT}/usr/local/bin/python
cp -pr /usr/local/lib/python3.9 ${CHROOT}/usr/local/lib
cp -p /usr/local/lib/libpython3.9.so.* ${CHROOT}/usr/local/lib
cp -p /usr/local/lib/libintl.so.7.0 ${CHROOT}/usr/local/lib
cp -p /usr/local/lib/libiconv.so.7.0 ${CHROOT}/usr/local/lib
cp -p /usr/lib/libpthread.so.* ${CHROOT}/usr/lib
cp -p /usr/lib/libutil.so.* ${CHROOT}/usr/lib
cp -p /usr/lib/libm.so.* ${CHROOT}/usr/lib
cp -p /usr/lib/libc.so.* ${CHROOT}/usr/lib
cp -p /usr/libexec/ld.so ${CHROOT}/usr/libexec
cp -p /usr/lib/libz.so.* ${CHROOT}/usr/lib
cp -p /usr/lib/libpthread.so.* ${CHROOT}/usr/lib
cp -p /usr/lib/libutil.so.* ${CHROOT}/usr/lib
cp -p /usr/lib/libm.so.* ${CHROOT}/usr/lib
cp -p /usr/lib/libssl.so* ${CHROOT}/usr/lib
cp -p /usr/lib/libcrypto.so* ${CHROOT}/usr/lib
cp -p /etc/pwd.db ${CHROOT}/etc
# build ld.hints.so file so python can find its libraries
chroot ${CHROOT} /sbin/ldconfig /usr/local/lib
Hope it helps.
-mark
--
Mark Willson
