Re: Guide for Configuring python(1) with httpd(8)
On Fri, Dec 23, 2022 at 07:57:56PM +, indivC wrote:
> However, the 'cgi' module is giving me trouble that I can't resolve.
> It simply won't import without errors.
>
> Why am I trying to import the 'cgi' module?
> What I want to do is pass data.
...
> The above is just a simple example that has one input field ('name').
> In order to grab the 'name' inputted by the user,
> I need to use the 'cgi' module.
Firstly, you don't actually _need_ to use the Python cgi module to write a cgi
program that handles input and output from the webserver such as form
submissions, it's just one way of doing it. CGI programs just read from
standard input and write to standard output. If you are doing this as a
learning experience, it would be much more educational to actually study the
format that the webserver uses to send the form data and write a simple parser
for it.
But it seems that most people these days want to take all of the shortcuts.
> When I run 'chroot /var/www htdocs/test/cgi-test.py',
> I get the below:
>
> File "/usr/local/lib/python3.9/email/header.py", line 14,
> in
> import binascii
> ImportError: Cannot load specified object
This is because the chroot environment is not fully set up. It doesn't
contain all of the files in all of the right locations for what you want to
do.
> I know, chroot is bad bad.
> I think once I can resolve this,
> I'll go back through your responses
> and attempt to move away from chroot and start using fastcgi.
Why not just start using FastCGI now? Honestly, you are just wasting your own
time by persuing the 'python interpreter in a chroot' method. Other people
have pointed this out. Especially since you are starting from scratch, and
not trying to run a piece of existing software that requires it.
I showed you how to get the python interpreter itself working in a chroot,
because that is useful general knowledge to have from a technical point of
view, and helps to explain in very simple terms how things work in a chroot.
But beyond just seeing it work and writing a hello world program, it's not
worth persuing.
> I tried to troubleshoot the above error,
> but I haven't gotten anywhere.
> My first thought was where is this module located,
> so I ran 'python3' to run the Python Interpreter
> and entered the below:
> >>> import binascii
> >>> binascii.__file__
> '/usr/local/lib/python3.9/lib-dynload/binascii.cpython-39.so'
>
> So that's the location of the module.
> It is located in the same path within '/var/www/'
> and it also has the correct permissions,
> but unsure what's the problem with importing it.
If you want to set up the chroot more thoroughly so that these extra Python
modules work, why don't you just use the script that Mark sent you:
https://marc.info/?l=openbsd-misc=167135242321424=2
I know you had a few problems with it:
> I removed the parts in the script
> that dealt with touching any folder path with 'run'.
> 'slowcgi.sock' is in '/var/www/run/'
> and I didn't want to mess with it.
> Also, it doesn't look like the script does anything with files
> in these folders, so it shouldn't matter that I omitted them.
>
> On the first run, it wasn't able to copy 'libiconv.so.7.0'.
> On my system, it's 'libiconv.so.7.1'.
This is because Mark's script was written for an older version of OpenBSD.
I tested it on a fresh OpenBSD 7.1 installation and it applied just fine
without any changes whatsoever. And I was able to use the python CGI module
in the chroot.
> Therefore, I updated that line in the script to 'libiconv.so.*'.
> This better matches how all the other lines are in the script.
> I'm not sure why this line and the one above it are different.
>
> Then, I ran the script again.
> However, I still get the same 'chroot' error:
> "ldconfig: /var/run/ld.so.hints.: No such file or directory"
Are you sure that /var is not full?
> It's like something is attempting to generate a pseudorandom file
> using 'ld.so.hints' as a base.
It is :-).
man 3 mktemp
man 3 mkstemp
> These pseudorandom filenames do not exist,
> but '/var/run/ld.so.hints' does,
> so I'm not sure why it doesn't just use that file directly.
Remember that OpenBSD is a multi-user system.
Like a lot of system programs, ldconfig does not overwrite it's files
directly when updating them. If it did then there would be a short period of
time where /var/run/ld.so.hints contained invalid or incomplete data. What
would happen if another program tried to access it at that moment? Or if the
system crashed?
The way to avoid this on basically any unix-like system is to write the
changed version to a new temporary file, then once it's written you rename the
temporary file to the real name, overwriting the old file.
The rename operation is atomic, I.E. it happens 'instantly', so there should
be no risk of the file becoming mangled.
If /var is not full, then you have probably made a configuration change that
you haven't told us about whilst trying to set
Re: Guide for Configuring python(1) with httpd(8)
--- Original Message ---
On Tuesday, December 20th, 2022 at 6:44 PM, Crystal Kolipe
wrote:
If you don't mind, I'd like to respond to some of this later.
I think I better understand some of my confusions,
but I'm not fully there yet.
Currently, I've had no issues displaying content.
I can display HTML from within a python file
and run python code to alter what's displayed in HTML.
All good stuff.
However, the 'cgi' module is giving me trouble that I can't resolve.
It simply won't import without errors.
Why am I trying to import the 'cgi' module?
What I want to do is pass data.
For instance, here's 'test.py':
#!/usr/local/bin/python3
print ('''
Name
Submit
''')
print(f"Your name is 'name_goes_here'")
The above is just a simple example that has one input field ('name').
In order to grab the 'name' inputted by the user,
I need to use the 'cgi' module.
The python code above doesn't contain anything
for the 'cgi' module as I can't import it.
I just thought showing an example might avoid any confusion.
As for what's the problem with the 'cgi' module,
below is a test file called 'cgi-test.py':
#!/usr/local/bin/python3
import cgi
When I run 'chroot /var/www htdocs/test/cgi-test.py',
I get the below:
File "/usr/local/lib/python3.9/email/header.py", line 14,
in
import binascii
ImportError: Cannot load specified object
I know, chroot is bad bad.
I think once I can resolve this,
I'll go back through your responses
and attempt to move away from chroot and start using fastcgi.
I tried to troubleshoot the above error,
but I haven't gotten anywhere.
My first thought was where is this module located,
so I ran 'python3' to run the Python Interpreter
and entered the below:
>>> import binascii
>>> binascii.__file__
'/usr/local/lib/python3.9/lib-dynload/binascii.cpython-39.so'
So that's the location of the module.
It is located in the same path within '/var/www/'
and it also has the correct permissions,
but unsure what's the problem with importing it.
Re: Guide for Configuring python(1) with httpd(8)
On Tue, Dec 20, 2022 at 02:01:03PM +, indivC wrote: > Crystal, > > I really appreciate the detailed explanations > and step by step instructions. > I was able to follow everything without a problem > and was able to finally access the python file from a web browser. Glad you've got it working! > On Monday, December 19th, 2022 at 11:07 AM, Crystal Kolipe > wrote: > > > # mkdir /var/www/usr/local/lib/pyton3.9 > > # mkdir /var/www/usr/local/include/pyton3.9 > > A slight correction to the lines above > in case anyone comes across this in the future. > The above lines should be: > mkdir -p /var/www/usr/local/lib/python3.9 > mkdir -p /var/www/usr/local/include/python3.9 Yes, you're right, you need to make the intermediate directories too. > > The /var/www/usr/local/lib path is not being searched for dynamic > > libraries when you try to run the python interpreter within the > > chroot. The easiest way to 'make it work' is to move the files > > you just copied to /var/www/usr/local/lib/ to /var/www/usr/lib/ > > instead. > > I think the first sentence makes sense to me. > While trying to search for a solution to the 'ld.so' error, > a lot of the solutions recommended two possible solutions. > The first was to add '/var/www/usr/local/lib' > to the library search path with > 'export LD_LIBRARY_PATH=/usr/lib:/var/www/usr/local/lib'. > The second was to attempt the same thing with > 'ldconfig /var/www/usr/local/lib'. > Neither of these seemed to work, so not sure why. It doesn't make any sense to add a path beginning with /var/www to the library search path from outside the chroot. By doing that, instead of allowing programs within the chroot to search the additional directory or directories, you are telling programs outside the chroot that they can do so. Obviously you will never have a program running outside the chroot that needs to load libraries from /var/www/...anything... Within the chroot, the path would be /usr/local/lib anyway, without the leading /var/www, and if you want to add that path to the library search path within the chroot, then you can run lconfig from within the chroot: # mkdir /var/www/sbin # mkdir -p /var/www/var/run # cp /sbin/ldconfig /var/www/sbin/ # chroot /var/www # sbin/ldconfig /usr/local/lib All this is doing is creating a /var/www/var/run/ld.so.hints file. > Also, I don't really understand why your solution worked. Because /usr/lib is always searched, unless you take specific steps to exclude it. On a normal system with hundreds or thousands of libraries, you probably wouldn't want everything in one directory. But a chroot environment is intended to be minimalist anyway, so it's really a matter of personal preference whether you create a /usr/local/lib directory for non-system libraries or not. > I understand why putting python(1) in chroot is a bad idea. > Therefore, what are the other possible options? Use FastCGI :-) This changes everything. Your python program now runs continuously instead of being launched repeatedly by the webserver for each incoming request. It communicates with the webserver using a socket, (usually a local socket), using the FastCGI protocol. This means that your program does not need to, (and should not), run within the webserver's chroot. It might run in it's own chroot, and it might also use other security features of OpenBSD such as unveil and pledge. It can do this _entirely separately_ from anything that httpd does. In this way, if your program is compromised, it does not have direct access to the memory space of the webserver, so it cannot change the contents of variables, run arbitrary code in the scope of httpd, etc, etc. It's also usually better for performance. Consider, for example, if your program reads a large database when it starts. The way you have things set up at the moment, it's going to be doing that read for every single request. With a single long-running invocation, which is what you usually have when using FastCGI, the database only needs to be read once. And of course you don't have the complicated set up of the chroot to deal with.
Re: Guide for Configuring python(1) with httpd(8)
Crystal, I really appreciate the detailed explanations and step by step instructions. I was able to follow everything without a problem and was able to finally access the python file from a web browser. On Monday, December 19th, 2022 at 11:07 AM, Crystal Kolipe wrote: > # mkdir /var/www/usr/local/lib/pyton3.9 > # mkdir /var/www/usr/local/include/pyton3.9 A slight correction to the lines above in case anyone comes across this in the future. The above lines should be: mkdir -p /var/www/usr/local/lib/python3.9 mkdir -p /var/www/usr/local/include/python3.9 > The /var/www/usr/local/lib path is not being searched for dynamic > libraries when you try to run the python interpreter within the > chroot. The easiest way to 'make it work' is to move the files > you just copied to /var/www/usr/local/lib/ to /var/www/usr/lib/ > instead. I think the first sentence makes sense to me. While trying to search for a solution to the 'ld.so' error, a lot of the solutions recommended two possible solutions. The first was to add '/var/www/usr/local/lib' to the library search path with 'export LD_LIBRARY_PATH=/usr/lib:/var/www/usr/local/lib'. The second was to attempt the same thing with 'ldconfig /var/www/usr/local/lib'. Neither of these seemed to work, so not sure why. Also, I don't really understand why your solution worked. > The first thing to understand is that there are several ways to > do what you want to do. Quite a lot of different ways, actually. > That's ONE way of doing it. Definitely NOT the best way for a > real application, but if you're just learning this then it's > probably the easiest^Wleast difficult way in. > Putting python in the chroot is ONE way of doing it. > > It's not the best way in general. But it might be the best way for > you, if you're trying to get an introduction to doing these things. I understand why putting python(1) in chroot is a bad idea. Therefore, what are the other possible options? When I was searching online, I couldn't find anything that didn't involve moving python(1) into chroot, so I'm not sure how to even find another possible solution, let alone a list of them analyzing the pros and cons of each. Again, I really appreciate your help. With it working now, I can start writing some python web apps. Just to be clear, this will not be on a production machine. It's on a non-networked machine that I'm just using to learn that has a web browser installed. However, even though my problem has been solved, I'd like to understand the proper way to configure this.
Re: Guide for Configuring python(1) with httpd(8)
On Mon, Dec 19, 2022 at 05:43:28AM +, indivC wrote:
> What I'm trying to do is display a python file
> that has imbedded HTML within a web browser.
...
> Within a browser, I want to be able to access this file
> and see 'Hello World' displayed.
OK, so you basically want to write a web application in Python.
It seems that you are doing this as an educational/learning
project, which was not obvious to me from your original email, and
judging from the replies you got, probably not obvious to other
people either. But no worries, let's look at what's needed.
> Where does this file need to be placed?
> What configuration changes need to be made to make this work?
The first thing to understand is that there are several ways to
do what you want to do. Quite a lot of different ways, actually.
> Currently, httpd(8) is running
> and I'm able to access HTML files via a web browser.
Good start. You've got static content working.
> I've seen mentions online that I needed to run slowcgi(8)
> in order to get this to work,
That's ONE way of doing it. Definitely NOT the best way for a
real application, but if you're just learning this then it's
probably the easiest^Wleast difficult way in.
> so I did 'rcctl enable slowcgi' and
> 'rcctl start slowcgi'.
If you've been reading stuff from different sources, some
confusion may have crept in about the whole cgi, and fastcgi stuff.
There is no real standard called 'slowcgi'. The original standard
was just called CGI, and for many years this is what webservers
implemented to run external programs that created dynamic content.
Later, the FastCGI standard was developed to avoid the shortcomings
of CGI, (such as the poor performance caused by needing to fork for
every new invocation of the external program), and webservers began
to implement the FastCGI standard.
In OpenBSD, the native httpd ONLY implements FastCGI. In order to
use the original CGI standard, a wrapper was created that invokes
the CGI program and passes it's output to httpd as if it was a
FastCGI program. That wrapper is called slowcgi.
> Then, I placed 'hello_world.py' in '/var/www/htdocs/test/'
> and then added the 'location' configuration to '/etc/httpd.conf'.
> Below is the full 'httpd.conf':
> server "test" {
> listen on 10.1.1.1 port 80
> root "/htdocs/test"
> location "*.py" {
> fastcgi
> root "/htdocs/test"
> }
> }
>
> types {
> include "/usr/share/misc/mime.types"
> }
That httpd.conf should work.
> The reason I was messing with python(1) in chroot
> was because that's what I was seeing online.
> If that doesn't need to be messed with, great.
> However, I don't know what I'd need to do to make this work.
Putting python in the chroot is ONE way of doing it.
It's not the best way in general. But it might be the best way for
you, if you're trying to get an introduction to doing these things.
Just DON'T try running a production server like this.
Basically, if you want to run _anything_ in a chroot, you need
to make sure that it has access to _all_ of the files that it
normally uses when run from outside the chroot.
I.E. you need to copy them from outside the chroot to inside.
For many simple programs, it is just a case of running ldd and
seeing what libraries it uses. But not always.
The python interpreter is a large and complicated piece of
software, and as you have discovered, just copying the libraries
that ldd reports as being needed is not enough.
The error you are getting:
> "ld.so: python3: can't load library 'libintl.so.7.0'"
Means that the specified library couldn't be found, or could
not be accessed.
But why? As you say:
> 'libintl.so.7.0' was one of the files that appeared in ldd(1).
> ldd(1) said the path was '/usr/local/lib/'
> and I've got the file copied to '/var/www/usr/local/lib/',
> so I'm unsure why python(1) is saying it can't load it.
Remember that the chroot is a minimalist environment. You have
not re-created the entire configuration of a normal OpenBSD install.
The /var/www/usr/local/lib path is not being searched for dynamic
libraries when you try to run the python interpreter within the
chroot. The easiest way to 'make it work' is to move the files
you just copied to /var/www/usr/local/lib/ to /var/www/usr/lib/
instead.
Now, when you enter the chroot and try to run your python program
you'll get further, (but it still won't work):
Could not find platform independent libraries
Could not find platform dependent libraries
Consider setting $PYTHONHOME to [:]
Python path configuration:
PYTHONHOME = (not set)
PYTHONPATH = (not set)
program name = '/usr/local/bin/python3'
isolated = 0
environment = 1
user site = 1
import site = 1
sys._base_executable = '/usr/local/bin/python3'
sys.base_prefix = '/usr/local'
sys.base_exec_prefix = '/usr/local'
sys.platlibdir = 'lib'
sys.executable = '/usr/local/bin/python3'
sys.prefix = '/usr/local'
sys.exec_prefix = '/usr/local'
sys.path
Re: Guide for Configuring python(1) with httpd(8)
Thanks for all the responses.
I'll respond to everyone (Omar, Crystal, Stuart, Bodie)
within this message to prevent sending out multiple messages.
It appears my message was significantly less clear than I thought.
My apologies.
Let me try again and
then I'll respond directly to messages further down.
What I'm trying to do is display a python file
that has imbedded HTML within a web browser.
The file is 'hello_world.py' and below are its contents:
#!/usr/local/bin/python3
print('Content-type:text/html\n')
print('')
print('')
print('Hello World')
print('')
print('\n')
Within a browser, I want to be able to access this file
and see 'Hello World' displayed.
Where does this file need to be placed?
What configuration changes need to be made to make this work?
Currently, httpd(8) is running
and I'm able to access HTML files via a web browser.
I've seen mentions online that I needed to run slowcgi(8)
in order to get this to work, so I did 'rcctl enable slowcgi' and
'rcctl start slowcgi'.
Then, I placed 'hello_world.py' in '/var/www/htdocs/test/'
and then added the 'location' configuration to '/etc/httpd.conf'.
Below is the full 'httpd.conf':
server "test" {
listen on 10.1.1.1 port 80
root "/htdocs/test"
location "*.py" {
fastcgi
root "/htdocs/test"
}
}
types {
include "/usr/share/misc/mime.types"
}
The reason I was messing with python(1) in chroot
was because that's what I was seeing online.
If that doesn't need to be messed with, great.
However, I don't know what I'd need to do to make this work.
On Sunday, December 18th, 2022 at 11:18 AM, Omar Polo wrote:
> not exactly, fastcgi is a binary protocol, whereas from what you're
> writing I'm assuming you're trying to run a CGI script written in
> python with slowcgi.
>
> (this is what I meant with "explain what you're trying to do" as there
> is a big difference between running a python web application and
> running a custom CGI script.)
Sorry. Hopefully what I have at the top of this message
makes it more clear.
I'm guessing the answer is I'm trying to run a python web application,
but honestly, I'm not sure.
When you say 'custom CGI script', that just confuses me.
I'm assuming that's more backend web development
where data is passed from the server via CGI (slowcgi/fastcgi?)
to python(1).
Whereas python web application
is more of the front end web development
that controls how the page itself looks.
Currently, I'm just trying to load a python file to display content.
> On pypi there is a 'fastcgi' library.
I'll have to take a look at this.
I didn't think I'd need to install an additional library
if I was just trying to display some HTML content.
On Sunday, December 18th, 2022 at 11:38 AM, Crystal Kolipe
wrote:
> It's probably about one day's work.
What may be one day's worth of work for you may be weeks/months
for me.
I don't forsee this being that easy for me.
On Sunday, December 18th, 2022 at 1:11 PM, Stuart Henderson
wrote:
> Surely the goal is to run some particular software and make it available
> via an httpd(8) frontend and not just run python(1) i.e. the Python
> interpreter?
Actually, I think this is exactly what I'm trying to do right now.
Maybe it's stupid to do this,
but I figured I'd start just with python.
After getting that to work and understanding it,
then maybe use additional software with python.
See my original message at the top that hopefully better explains
what I'm trying to do.
>fastcgi
> is more common with other languages but also possible with Python and
> would often be a better fit than trying to run Python directly in a chroot
> via httpd->fastcgi->slowcgi->python(1).
Any guides, resources, or rough instructions on how to do this?
On Sunday, December 18th, 2022 at 3:21 PM, Bodie wrote:
> As was already pointed out by others it may be good to know what
> are you trying to achieve actually aka why you are trying to put
> python in chroot. Especially as you said it works with perl fine
> and that one is in base.
Apologies. Hopefully my message at the top is more clear now.
Re: Guide for Configuring python(1) with httpd(8)
On 18.12.2022 08:07, indivC wrote: Can anyone provide a guide for this or rough instructions? I'm running httpd(8) and trying to utilize a python(1) script with an html file. I've got this working using perl(1). However, it doesn't work with python(1) when following the same steps. As was already pointed out by others it may be good to know what are you trying to achieve actually aka why you are trying to put python in chroot. Especially as you said it works with perl fine and that one is in base. My python(1) version is 3.9.15 My OpenBSD version is 7.2 stable. First, I use ldd(1) to determine what files I need to copy for python(1). Second, in '/var/www/', I create all the folder paths from the ldd(1) output. Third, I change the ownership of all the folders to be 'www:www'. Then, I copy all the files from the ldd(1) output into their respective folders. Lastly, I change the permissions of all these files to 750. At this point, before doing anything with httpd(8), I try testing to ensure everything is correct with python(1). If I run 'python3 /var/www/htdocs/test/test.py', the script runs. However, if I run 'chroot /var/www htdocs/test/test.py', I get the following error: "ld.so: python3: can't load library 'libintl.so.7.0'" 'libintl.so.7.0' was one of the files that appeared in ldd(1). ldd(1) said the path was '/usr/local/lib/' and I've got the file copied to '/var/www/usr/local/lib/', so I'm unsure why python(1) is saying it can't load it. What's a little strange is if I run the 'chroot' command from above several times in a row, I'll see the same error for 'libpython3.9.so.0.0' as well. ldd(1) also showed this file and I also have it, so I'm not really sure what the problem is. I've seen some posts indicate that I also need to copy '/sbin/ldconfig' to '/var/www/'. Then, run 'chroot /var/www sbin/ldconfig /usr/local/lib/'. When I do that, I get the following error: "ldconfig: /var/run/ld.so.hints.: No such file or directory" In the above error, '' seems to be some pseudorandom value. If I run the command multiple times, that value changes every time. For instance, one value I got is 'IkB2akBOKX'. Maybe the steps to configure this are a little different for python(1) when compared to perl(1). This is why I was hoping someone could provide a link to a guide or provide some general steps for configuring this. Currently, I've just been trying to piece together what I've been able to find online. Thanks.
Re: Guide for Configuring python(1) with httpd(8)
On 2022-12-18, indivC wrote: > On Sunday, December 18th, 2022 at 9:04 AM, Omar Polo > wrote: > >> instead of asking how to do X so that you can do Y, ask directly how >> to do Y. > > I did. The first line of my message was > "Can anyone provide a guide for this or rough instructions?", > which is in reference to the subject > "Guide for Configuring python(1) with httpd(8)". Surely the goal is to run some particular software and make it available via an httpd(8) frontend and not just run python(1) i.e. the Python interpreter? > Did i precede to explain > how I was trying to attempt to accomplish Y with X? > Yes, but I don't see why that would be a problem. > I feel like it's better for users to actually attempt to try > and solve their problems then not to try at all. Yes but when you would like help, describe and ask about the base problem rather than something further down the line of your idea if how to do it. >> Why do you need python at all in the chroot? Installing all the >> needed files (and keeping them up-to-date!) manually in a chroot is a >> pain. > > I completely agree. > Virtually every solution I've seen online does this. Every solution for "run stuff in chroot" does this, but there are other more common ways to expose Python software to web clients. scgi is possibly the most common but openbsd's base httpd(8) doesn't support that. fastcgi is more common with other languages but also possible with Python and would often be a better fit than trying to run Python directly in a chroot via httpd->fastcgi->slowcgi->python(1). >> Since httpd speaks fastcgi, why not write some python code that >> accepts the requests over fastcgi? (assuming this is what you're >> trying to do, but you didn't tell.) > > I believe that is exactly what I'm trying to do. > The end goal is to be able access a python(1) file from httpd(8). > My understanding is you have to configure slowcgi(8), > which utilizes fastcgi, within httpd(8). The process accepting fastcgi requests would often run (or at least be started from) *outside* the chroot. Separate process waiting and listening for requests, not something executed by the httpd process. Another alternative is to use slowcgi but disable the chroot (-p /). Is chroot really buying you much if you provide an environment making it easier to run things inside that chroot anyway? -- Please keep replies on the mailing list.
Re: Guide for Configuring python(1) with httpd(8)
On Sun, Dec 18, 2022 at 12:18:32PM +0100, Omar Polo wrote: > On pypi there is a 'fastcgi' library. it's not packaged on OpenBSD > and I can't asses how good it is Alternatively, just write a fastcgi handler from scratch - the protocol is fairly simple and fully documented. Httpd only implements a minimalist subset of fastcgi functions anyway, so any fastcgi implementation that's designed to work with it only needs to implement the same subset. It's probably about one day's work.
Re: Guide for Configuring python(1) with httpd(8)
On 2022/12/18 10:23:39 +, indivC wrote:
> On Sunday, December 18th, 2022 at 9:04 AM, Omar Polo
> wrote:
> > Since httpd speaks fastcgi, why not write some python code that
> > accepts the requests over fastcgi? (assuming this is what you're
> > trying to do, but you didn't tell.)
>
> I believe that is exactly what I'm trying to do.
> The end goal is to be able access a python(1) file from httpd(8).
> My understanding is you have to configure slowcgi(8),
> which utilizes fastcgi, within httpd(8).
> I've been able to configure that without any problems using perl(1),
> but not with python(1).
not exactly, fastcgi is a binary protocol, whereas from what you're
writing I'm assuming you're trying to run a CGI script written in
python with slowcgi.
(this is what I meant with "explain what you're trying to do" as there
is a big difference between running a python web application and
running a custom CGI script.)
there are less ugly hacks (IMHO) than one can do with slowcgi instead
of installing huge things like scripting languages inside the /var/www
chroot, but...
> With python(1), because the problem is occurring
> before I even get to httpd(8), I left that out of the message.
> If I attempt to access the python(1) file currently,
> I still see the same 'ld.so' error message I mentioned before.
> Therefore, I was trying to leave out httpd(8)
> and slowcgi(8) configuration as that isn't where the problem lies.
>
> As mentioned before, if you have a better solution for this,
> please share.
> Any links you can provide for step by step instructions
> or general steps on how to accomplish this would be helpful.
>
> Thanks.
...I think that instead of installing python in the /var/www chroot
(plus all the dependencies you'd need) if you really want to use httpd
and write your stuff in python you may have your python script talk
fastcgi instead.
On pypi there is a 'fastcgi' library. it's not packaged on OpenBSD
and I can't asses how good it is, I'm not reccomanding it (just did a
random search on the web and was the first result), but at least seems
to work: (modeled after the example code in the github repo)
#!/usr/bin/env python
# hello.py
from fastcgi import *
from socketserver import TCPServer
class MyHandler(FcgiHandler):
def handle(self):
self.print('Content-type: text/plain\n')
self.print("Hello, world!")
addr = ('localhost', )
with TCPServer(addr, MyHandler) as srv:
srv.serve_forever()
and here's the matching httpd.conf bit
server "localhost" {
listen on * port 80
fastcgi socket tcp localhost
}
this has the advantage of not needing to fork one process per request
like CGI would do, but you need to secure your application by yourself
(running it as a different user for starters -- no idea if you can do
fancier things with python.)
Re: Guide for Configuring python(1) with httpd(8)
On Sunday, December 18th, 2022 at 8:38 AM, Mark Willson > This is the script I use to set up python for httpd: I looked over the script and it seems to do pretty much what I've already done. It copies some additional files that I didn't copy. My first attempt was just to copy over the additional files that I had not previously copied that are in your script. This didn't change anything for me. I then decided to run your script directly. I removed the parts in the script that dealt with touching any folder path with 'run'. 'slowcgi.sock' is in '/var/www/run/' and I didn't want to mess with it. Also, it doesn't look like the script does anything with files in these folders, so it shouldn't matter that I omitted them. On the first run, it wasn't able to copy 'libiconv.so.7.0'. On my system, it's 'libiconv.so.7.1'. Therefore, I updated that line in the script to 'libiconv.so.*'. This better matches how all the other lines are in the script. I'm not sure why this line and the one above it are different. Then, I ran the script again. However, I still get the same 'chroot' error: "ldconfig: /var/run/ld.so.hints.: No such file or directory" So I'm not sure why I'm getting different results than you. It's like something is attempting to generate a pseudorandom file using 'ld.so.hints' as a base. These pseudorandom filenames do not exist, but '/var/run/ld.so.hints' does, so I'm not sure why it doesn't just use that file directly.
Re: Guide for Configuring python(1) with httpd(8)
On Sunday, December 18th, 2022 at 9:04 AM, Omar Polo wrote: > instead of asking how to do X so that you can do Y, ask directly how > to do Y. I did. The first line of my message was "Can anyone provide a guide for this or rough instructions?", which is in reference to the subject "Guide for Configuring python(1) with httpd(8)". Did i precede to explain how I was trying to attempt to accomplish Y with X? Yes, but I don't see why that would be a problem. I feel like it's better for users to actually attempt to try and solve their problems then not to try at all. > Why do you need python at all in the chroot? Installing all the > needed files (and keeping them up-to-date!) manually in a chroot is a > pain. I completely agree. Virtually every solution I've seen online does this. If there's a better way, I'm all ears. I think it's reasonable to assume if someone is attempting to do something and you believe you have a much better way of accomplishing it, then share it. I never said this is the only way to do it or that this is the only way I want to accomplish this problem. I just shared what I've been trying to get to work, which is based on the information I've found online. > Since httpd speaks fastcgi, why not write some python code that > accepts the requests over fastcgi? (assuming this is what you're > trying to do, but you didn't tell.) I believe that is exactly what I'm trying to do. The end goal is to be able access a python(1) file from httpd(8). My understanding is you have to configure slowcgi(8), which utilizes fastcgi, within httpd(8). I've been able to configure that without any problems using perl(1), but not with python(1). With python(1), because the problem is occurring before I even get to httpd(8), I left that out of the message. If I attempt to access the python(1) file currently, I still see the same 'ld.so' error message I mentioned before. Therefore, I was trying to leave out httpd(8) and slowcgi(8) configuration as that isn't where the problem lies. As mentioned before, if you have a better solution for this, please share. Any links you can provide for step by step instructions or general steps on how to accomplish this would be helpful. Thanks.
Re: Guide for Configuring python(1) with httpd(8)
instead of asking how to do X so that you can do Y, ask directly how to do Y. Why do you need python at all in the chroot? Installing all the needed files (and keeping them up-to-date!) manually in a chroot is a pain. Since httpd speaks fastcgi, why not write some python code that accepts the requests over fastcgi? (assuming this is what you're trying to do, but you didn't tell.) On 2022/12/18 07:07:20 +, indivC wrote: > Can anyone provide a guide for this or rough instructions? > I'm running httpd(8) and trying to utilize a python(1) script > with an html file. > I've got this working using perl(1). > However, it doesn't work with python(1) when following the same steps. > > My python(1) version is 3.9.15 > My OpenBSD version is 7.2 stable. > > First, I use ldd(1) > to determine what files I need to copy for python(1). > Second, in '/var/www/', > I create all the folder paths from the ldd(1) output. > Third, I change the ownership of all the folders to be 'www:www'. > Then, I copy all the files from the ldd(1) output > into their respective folders. > Lastly, I change the permissions of all these files to 750. > > At this point, before doing anything with httpd(8), > I try testing to ensure everything is correct with python(1). > If I run 'python3 /var/www/htdocs/test/test.py', the script runs. > However, if I run 'chroot /var/www htdocs/test/test.py', > I get the following error: > "ld.so: python3: can't load library 'libintl.so.7.0'" > > 'libintl.so.7.0' was one of the files that appeared in ldd(1). > ldd(1) said the path was '/usr/local/lib/' > and I've got the file copied to '/var/www/usr/local/lib/', > so I'm unsure why python(1) is saying it can't load it. > > What's a little strange > is if I run the 'chroot' command from above several times in a row, > I'll see the same error for 'libpython3.9.so.0.0' as well. > ldd(1) also showed this file and I also have it, > so I'm not really sure what the problem is. > > I've seen some posts indicate > that I also need to copy '/sbin/ldconfig' to '/var/www/'. > Then, run 'chroot /var/www sbin/ldconfig /usr/local/lib/'. > When I do that, I get the following error: > "ldconfig: /var/run/ld.so.hints.: No such file or directory" > > In the above error, '' seems to be some pseudorandom value. > If I run the command multiple times, that value changes every time. > For instance, one value I got is 'IkB2akBOKX'. > > Maybe the steps to configure this are a little different for python(1) > when compared to perl(1). > This is why I was hoping someone could provide a link to a guide > or provide some general steps for configuring this. > Currently, I've just been trying to piece together > what I've been able to find online. > > Thanks.
Re: Guide for Configuring python(1) with httpd(8)
> -Original Message-
> From: owner-m...@openbsd.org On Behalf Of indivC
> Sent: 18 December 2022 07:07
> To: misc@openbsd.org
> Subject: Guide for Configuring python(1) with httpd(8)
>
> Can anyone provide a guide for this or rough instructions?
> I'm running httpd(8) and trying to utilize a python(1) script
> with an html file.
[snipped]
> This is why I was hoping someone could provide a link to a guide
> or provide some general steps for configuring this.
> Currently, I've just been trying to piece together
> what I've been able to find online.
>
> Thanks.
Hi indivC,
This is the script I use to set up python for httpd:
#!/bin/sh
#
# Enable Python3 to operate in a chrooted environment (for httpd)
#
export CHROOT=/var/www
rm -rf ${CHROOT}/usr ${CHROOT}/var ${CHROOT}/etc ${CHROOT}/sbin \
${CHROOT}/run ${CHROOT}/logs
mkdir -p ${CHROOT}/usr/local/bin ${CHROOT}/usr/lib ${CHROOT}/usr/libexec \
${CHROOT}/sbin ${CHROOT}/var/run ${CHROOT}/etc \
${CHROOT}/usr/local/lib ${CHROOT}/run ${CHROOT}/logs
chown www ${CHROOT}/usr/local/bin ${CHROOT}/usr/lib ${CHROOT}/usr/libexec \
${CHROOT}/sbin ${CHROOT}/var/run ${CHROOT}/etc \
${CHROOT}/usr/local/lib ${CHROOT}/run ${CHROOT}/logs
cp -p /sbin/ldconfig ${CHROOT}/sbin
cp -p /usr/local/bin/python3.9 ${CHROOT}/usr/local/bin/python
cp -pr /usr/local/lib/python3.9 ${CHROOT}/usr/local/lib
cp -p /usr/local/lib/libpython3.9.so.* ${CHROOT}/usr/local/lib
cp -p /usr/local/lib/libintl.so.7.0 ${CHROOT}/usr/local/lib
cp -p /usr/local/lib/libiconv.so.7.0 ${CHROOT}/usr/local/lib
cp -p /usr/lib/libpthread.so.* ${CHROOT}/usr/lib
cp -p /usr/lib/libutil.so.* ${CHROOT}/usr/lib
cp -p /usr/lib/libm.so.* ${CHROOT}/usr/lib
cp -p /usr/lib/libc.so.* ${CHROOT}/usr/lib
cp -p /usr/libexec/ld.so ${CHROOT}/usr/libexec
cp -p /usr/lib/libz.so.* ${CHROOT}/usr/lib
cp -p /usr/lib/libpthread.so.* ${CHROOT}/usr/lib
cp -p /usr/lib/libutil.so.* ${CHROOT}/usr/lib
cp -p /usr/lib/libm.so.* ${CHROOT}/usr/lib
cp -p /usr/lib/libssl.so* ${CHROOT}/usr/lib
cp -p /usr/lib/libcrypto.so* ${CHROOT}/usr/lib
cp -p /etc/pwd.db ${CHROOT}/etc
# build ld.hints.so file so python can find its libraries
chroot ${CHROOT} /sbin/ldconfig /usr/local/lib
Hope it helps.
-mark
--
Mark Willson
