Re: leaking rcache opens in gss_accept_sec_context
On Jul 20, 2011, at 1:07 AM, Greg Hudson wrote: On Tue, 2011-07-19 at 16:21 -0400, Benjamin Coddington wrote: >> gss_acquire_cred >> gss_accept_sec_context >> gss_export_lucid_sec_context >> gss_delete_sec_context > >> I found that before we got to gss_delete_sec_context(), we had already >> tried to clean up the context in gss_krb5_export_lucid_sec_context() >> -> krb5_gss_delete_sec_context(), which fails with G_VALIDATE_FAILED. >> It also sets the context to GSS_C_NO_CONTEXT, so once we get to >> gss_delete_sec_context(), context validation fails there too. > > Aha. Yes, that's the bug you found a reference to. (And thank you for > explaining why that bug wasn't resulting in gssd crashes for everyone in > previous releases. I had forgotten about the pointer validation code.) > I've attached the patch which is due for krb5 1.9.2. > > gss_delete_sec_context should be unnecessary when > gss_export_lucid_sec_context succeeds. Of course, it's harmless given > the way GSS handles contexts (nulling out the pointer when they are > released). > > Thank you, Greg. I can confirm that this fixes the problem we were seeing. It also fixes a leak when running without '-n', which was less obvious because we didn't open a new handle to the rcache each time. Ben Kerberos mailing list Kerberos@mit.edu https://mailman.mit.edu/mailman/listinfo/kerberos
Re: leaking rcache opens in gss_accept_sec_context
On Tue, 2011-07-19 at 16:21 -0400, Benjamin Coddington wrote: > gss_acquire_cred > gss_accept_sec_context > gss_export_lucid_sec_context > gss_delete_sec_context > I found that before we got to gss_delete_sec_context(), we had already > tried to clean up the context in gss_krb5_export_lucid_sec_context() > -> krb5_gss_delete_sec_context(), which fails with G_VALIDATE_FAILED. > It also sets the context to GSS_C_NO_CONTEXT, so once we get to > gss_delete_sec_context(), context validation fails there too. Aha. Yes, that's the bug you found a reference to. (And thank you for explaining why that bug wasn't resulting in gssd crashes for everyone in previous releases. I had forgotten about the pointer validation code.) I've attached the patch which is due for krb5 1.9.2. gss_delete_sec_context should be unnecessary when gss_export_lucid_sec_context succeeds. Of course, it's harmless given the way GSS handles contexts (nulling out the pointer when they are released). commit 1d72f6deeb2a8445567228de6495264112294223 Author: ghudson Date: Mon May 9 17:28:07 2011 + ticket: 6908 subject: Delete sec context properly in gss_krb5_export_lucid_sec_context target_version: 1.9.2 tags: pullup Since r21690, gss_krb5_export_lucid_sec_context() has been passing a union context to krb5_gss_delete_sec_context(), causing a crash as the krb5 routine attempts to interpret a union context structure as a krb5 GSS context. Call the mechglue gss_delete_sec_context instead. git-svn-id: svn://anonsvn.mit.edu/krb5/trunk@24917 dc483132-0cff-0310-8789-dd5450dbe970 diff --git a/src/lib/gssapi/krb5/krb5_gss_glue.c b/src/lib/gssapi/krb5/krb5_gss_glue.c index bc3b7c7..0035d4f 100644 --- a/src/lib/gssapi/krb5/krb5_gss_glue.c +++ b/src/lib/gssapi/krb5/krb5_gss_glue.c @@ -196,7 +196,7 @@ gss_krb5_export_lucid_sec_context(OM_uint32 *minor_status, /* Clean up the context state (it is an error for * someone to attempt to use this context again) */ -(void)krb5_gss_delete_sec_context(minor_status, context_handle, NULL); +(void)gss_delete_sec_context(minor_status, context_handle, NULL); *context_handle = GSS_C_NO_CONTEXT; generic_gss_release_buffer_set(&minor, &data_set); Kerberos mailing list Kerberos@mit.edu https://mailman.mit.edu/mailman/listinfo/kerberos
Re: leaking rcache opens in gss_accept_sec_context
On Jul 14, 2011, at 9:40 AM, Greg Hudson wrote: > On Wed, 2011-07-13 at 15:33 -0400, Benjamin Coddington wrote: >> Anyway, calling gss_accept_sec_context this way allows svcgssd to >> create a context for any requested service name -- but the problem we >> found is that svcgssd opens the kerberos replay cache for every >> context/cred created, eventually reaching ulimit. The files are never >> closed, and every so often the rcache is removed and re-written, so >> the handles themselves are to deleted files. > > The files should be closed in gss_delete_sec_context() (via > gssint_delete_internal_sec_context -> krb5_gss_delete_sec_context -> > krb5_auth_con_free -> krb5_rc_close). Is it possible that svcgssd is > leaking the security contexts? > > There is a behavior difference here depending on how the server invokes > gss_accept_sec_context(). If the server imports a name and acquires > credentials, the rcache handle is associated with the credential object. > If the server uses the default credentials or name, the rcache handle is > associated with the security context. If svcgssd leaks security > contexts but not credentials, you would only see the rcache leak when -n > is used. It shouldn't leak -- it roughly loops on gss_acquire_cred gss_accept_sec_context gss_export_lucid_sec_context gss_delete_sec_context I found that before we got to gss_delete_sec_context(), we had already tried to clean up the context in gss_krb5_export_lucid_sec_context() -> krb5_gss_delete_sec_context(), which fails with G_VALIDATE_FAILED. It also sets the context to GSS_C_NO_CONTEXT, so once we get to gss_delete_sec_context(), context validation fails there too. I found your message to K.C. in May (http://marc.info/?t=13047050791&r=1&w=2). Do you have a patch we could try? Ben Kerberos mailing list Kerberos@mit.edu https://mailman.mit.edu/mailman/listinfo/kerberos
Re: leaking rcache opens in gss_accept_sec_context
On Wed, 2011-07-13 at 15:33 -0400, Benjamin Coddington wrote: > Anyway, calling gss_accept_sec_context this way allows svcgssd to > create a context for any requested service name -- but the problem we > found is that svcgssd opens the kerberos replay cache for every > context/cred created, eventually reaching ulimit. The files are never > closed, and every so often the rcache is removed and re-written, so > the handles themselves are to deleted files. The files should be closed in gss_delete_sec_context() (via gssint_delete_internal_sec_context -> krb5_gss_delete_sec_context -> krb5_auth_con_free -> krb5_rc_close). Is it possible that svcgssd is leaking the security contexts? There is a behavior difference here depending on how the server invokes gss_accept_sec_context(). If the server imports a name and acquires credentials, the rcache handle is associated with the credential object. If the server uses the default credentials or name, the rcache handle is associated with the security context. If svcgssd leaks security contexts but not credentials, you would only see the rcache leak when -n is used. Kerberos mailing list Kerberos@mit.edu https://mailman.mit.edu/mailman/listinfo/kerberos
leaking rcache opens in gss_accept_sec_context
I am working on a linux NFS cluster that requires a single svcgssd to establish contexts under multiple service names. In this scenario, svcgssd can be started with "-n" so that it acquires creds at context creation. The behavior with "-n" is to call gss_accept_sec_context() with a NULL verifier_cred_handle instead of a gss_cred that was created beforehand with gss_acquire_cred(). I believe that the NULL verifier_cred_handle causes the kerberos code to try to create creds - perhaps by trying to create a cred for each principal in the keytab.. I am not an expert in the kerberos code. Anyway, calling gss_accept_sec_context this way allows svcgssd to create a context for any requested service name -- but the problem we found is that svcgssd opens the kerberos replay cache for every context/cred created, eventually reaching ulimit. The files are never closed, and every so often the rcache is removed and re-written, so the handles themselves are to deleted files. Is it advisable to call gss_accept_sec_context with a NULL verifier_cred_handle, and if so -- what can we do to release resources/close rcache each time? Ben Kerberos mailing list Kerberos@mit.edu https://mailman.mit.edu/mailman/listinfo/kerberos
