Re: (forw) [dfries@mail.win.org: Bug#161253: apache2: shared memory cleanup]

2002-09-20 Thread Aaron Bannert

On Wed, Sep 18, 2002 at 10:24:43AM +0100, Thom May wrote:
 Shared memory (as used by at least the ssl module) needs to rethink
 how they go about cleaning it up.  Currently kill -9 the apache
 processes or it would appear that even /etc/init.d/apache2 reload will
 cause apache to fail to start because the shared memory segment can't
 be created since it already exists.

Any time you use SIGKILL you're going to have to manually clean up
semaphores, segments, and other shared resources. The solution to
that is to not use kill -9.

 There are a couple solutions to this.  If the shared memory segment
 already exists just attach to it, but this has security issues.

It's not so much about security as it is about not knowing if there
is a current generation of httpd processes still using it or if it's
safe to jump right in and start reusing the same segment. I can't
think of any robust ways to detect this.

 If the shared memory segment already exists pick another magic number
 and try again, this could eventually leave many unused shared memory
 regious laying around.

At some point you will run out of system resources. The right thing
to do is to not use kill -9 and if you must, to make sure you clean
up your shared resources.

An alternative, since Debian/Linux has a robust VM system, would be
to make sure Apache uses an mmap()-based shared memory syscall. This
would take some tweaking of the autoconf scripts.

 This is how I've done it in the past.  Allocate the shared memory
 segment, attach to it, delete it.  It will stick around until the last
 process detaches.  We just need to use the shmid to attach to our
 segment instead of using the key.

I tried this when I was writing the new apr_shm.h stuff, but the problem
is once you mark a segment for deletion, no new processes can attach
to it (including children, if I remember correctly).

-aaron



(forw) [dfries@mail.win.org: Bug#161253: apache2: shared memory cleanup]

2002-09-18 Thread Thom May

I'm forwarding this debian bug onto the dev list for discussion and
comments.
please can the cc line ([EMAIL PROTECTED]) be maintained so the bug
gets a record of this?
Cheers,
-Thom
-- 
Thom May - [EMAIL PROTECTED]

 robster SHIT
 robster woody doesnt support devfs
 willy devfs is FUcking Shit
 liiwi willy: quick now! tell it to stop before it gets itself dirty!

---BeginMessage---

Package: apache2
Version: N/A; reported 2002-09-17
Severity: normal
Tags: patch

Shared memory (as used by at least the ssl module) needs to rethink
how they go about cleaning it up.  Currently kill -9 the apache
processes or it would appear that even /etc/init.d/apache2 reload will
cause apache to fail to start because the shared memory segment can't
be created since it already exists.

There are a couple solutions to this.  If the shared memory segment
already exists just attach to it, but this has security issues.

If the shared memory segment already exists pick another magic number
and try again, this could eventually leave many unused shared memory
regious laying around.

This is how I've done it in the past.  Allocate the shared memory
segment, attach to it, delete it.  It will stick around until the last
process detaches.  We just need to use the shmid to attach to our
segment instead of using the key.

Here is a patch to fix it.  I only have Unix so I couldn't test the
other platforms, something will have to be done about OS/2 and the
rest.

Index: modules/ssl/ssl_scache_shmcb.c
===
RCS file: 
/mnt/david/debian/apache2/apache2-2.0.40/build-tree/cvs_cache/httpd-2.0.40/modules/ssl/ssl_scache_shmcb.c,v
retrieving revision 1.1.1.1
diff -u -r1.1.1.1 ssl_scache_shmcb.c
--- modules/ssl/ssl_scache_shmcb.c  17 Sep 2002 15:49:50 -  1.1.1.1
+++ modules/ssl/ssl_scache_shmcb.c  17 Sep 2002 19:54:26 -
@@ -385,7 +385,30 @@
 }
 shm_segment = apr_shm_baseaddr_get(mc-pSessionCacheDataMM);
 shm_segsize = apr_shm_size_get(mc-pSessionCacheDataMM);
-
+/* We want the shared memory segment to go away as soon as the
+ * program exists.  This is done by marking it as destroyed
+ * which also removes the name-based shared memory lookup key
+ * attach, which is fine since shm access is preserved accross
+ * fork calls.  If the segment doesn't go away when we exit, we
+ * waste memory and can't restart since creating a memory
+ * segment that already exists fails.
+ */
+#if 1
+if(!apr_shm_mark_destroyed(mc-pSessionCacheDataMM))
+{
+   /* The worst thing we could do now is exit, since we know
+* the shared memory segment won't be cleaned up and
+* apache won't be able to start next time.  Just hope
+* it will be properly cleaned up when apache gracefully
+* exits.
+*/
+char buf[100];
+ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, s,
+ Cannot makr shared memory for destruction: (%d)%s, rv,
+ apr_strerror(rv, buf, sizeof(buf)));
+}
+#endif
+   
 ap_log_error(APLOG_MARK, APLOG_DEBUG, 0, s,
  shmcb_init allocated % APR_SIZE_T_FMT 
   bytes of shared memory,
Index: srclib/apr/include/apr_shm.h
===
RCS file: 
/mnt/david/debian/apache2/apache2-2.0.40/build-tree/cvs_cache/httpd-2.0.40/srclib/apr/include/apr_shm.h,v
retrieving revision 1.1.1.1
diff -u -r1.1.1.1 apr_shm.h
--- srclib/apr/include/apr_shm.h17 Sep 2002 15:49:58 -  1.1.1.1
+++ srclib/apr/include/apr_shm.h17 Sep 2002 19:54:57 -
@@ -109,6 +109,15 @@
  apr_pool_t *pool);
 
 /**
+ * Mark a shared memory segment for destruction when all
+ * processes detach from it.  Currently this prevents
+ * processes from attaching, this could change if we
+ * kept track of the segment id.
+ * @param m The shared memory segment structure to mark for destruction.
+ */
+APR_DECLARE(apr_status_t) apr_shm_mark_destroyed(apr_shm_t *m);
+
+/**
  * Destroy a shared memory segment and associated memory.
  * @param m The shared memory segment structure to destroy.
  */
Index: srclib/apr/shmem/unix/shm.c
===
RCS file: 
/mnt/david/debian/apache2/apache2-2.0.40/build-tree/cvs_cache/httpd-2.0.40/srclib/apr/shmem/unix/shm.c,v
retrieving revision 1.1.1.1
diff -u -r1.1.1.1 shm.c
--- srclib/apr/shmem/unix/shm.c 17 Sep 2002 15:49:57 -  1.1.1.1
+++ srclib/apr/shmem/unix/shm.c 17 Sep 2002 19:57:10 -
@@ -109,11 +109,40 @@
 
 /* Indicate that the segment is to be destroyed as soon
  * as all processes have detached. This also disallows any
- * new attachments to the segment. */
+ * new attachments to the segment using the named based
+* lookup key, we could however use the shared memory id
+* and attach 

Re: (forw) [dfries@mail.win.org: Bug#161253: apache2: shared memory cleanup]

2002-09-18 Thread Greg Stein

On Wed, Sep 18, 2002 at 10:24:43AM +0100, Thom May wrote:
...
 please can the cc line ([EMAIL PROTECTED]) be maintained so the bug
 gets a record of this?

Well, dunno about that. Since this list obeys Ken's nefarious insistence on
Reply-To munging, who knows what'll happen when people reply?

ducks

:-)

-- 
Greg Stein, http://www.lyra.org/