On Mon, 2015-04-27 at 18:41 +0300, Alexander Bokovoy wrote: > On Mon, 27 Apr 2015, Simo Sorce wrote: > >On Mon, 2015-04-27 at 15:04 +0300, Alexander Bokovoy wrote: > >> On Mon, 27 Apr 2015, Jakub Hrozek wrote: > >> >On Sun, Apr 26, 2015 at 06:17:21PM -0400, Simo Sorce wrote: > >> >> Very nice writeup! > >> >> A few comments.. > >> >> > >> >> On Sun, 2015-04-26 at 21:22 +0200, Jakub Hrozek wrote: > >> >> > Hi, > >> >> > > >> >> > I prepared design page for the one-way trusts feature: > >> >> > https://fedorahosted.org/sssd/wiki/DesignDocs/OneWayTrusts > >> >> > > >> >> > The text is copied below as well for your convenience. I have a couple > >> >> > of questions to discuss: > >> >> > - Is it worth it to put the keytabs into a new subdirectory so > >> >> > that > >> >> > a special SELinux label can be applied? I think yes, since then > >> >> > we'd have additional layer of protection of the trust secrets > >> >> > >> >> It is not clear to me what would be the advantage, who are you trying to > >> >> protect against ? > >> >> Adding different labels makes sense only if sssd_be has a different > >> >> label from the other sssd processes, but then you can simply have > >> >> selinux policies that label accordingly to the parent process in the > >> >> same directory, no? > >> > > >> >Yes, I tried to future-proof the directory structure against different > >> >sssd processes that need access to the database (responders) but not the > >> >keytabs, because I suspect we won't think about the directory structure > >> >again once we set it up :-). And a single separate directory just seemed > >> >easier to label with a new context than several files. > >> > > >> >In the end, I don't care that much as long as the keytabs have different > >> >contexts that the responders can't access. > >> Given that we want to have one keytab per forest trust and naming > >> keytabs after forest trust name, we can live with a single directory. It > >> would be good, indeed, to have it separated from the rest of sssd. > >> > >> > >> >> > >> >> > - Is there any reasonable fallback beyond re-fetching the keytab > >> >> > we > >> >> > can do? I can't think of any.. > >> >> > >> >> Be careful with refetches and so we do not want to end up in infinite > >> >> loops, where the keytab can be fetched but still operation fails for > >> >> whatever reason. > >> > > >> >Sure, I will add a more explicit note once we agree what the mechanism > >> >would be. > >> Another thing to remember is a potential need to limit enctypes you'd be > >> requesting because camellia ciphers are not know to AD and might cause > >> issues at some point. > > > >SSSD should not be in the business of *creating* keytabs, it should only > >be allowed to retrieve a precreated key, so SSSD shouldn't care about > >enctypes, it will get only those that the FreeIPA code stored in the key > >in LDAP. > So this means ipasam needs to limit enctypes when asking for the keys.
Yes it should only ask for encrypts that the AD server on the other side understand, but only for good measure. The key used is alwasy determined by the KDC (AD in this case) so having additional keys in the keytab is not a problem. A problem would rather be to miss enctypes, as the KDC could decide to encode a ticket/TGT in one of the missing enctypes then and we would not be able to decrypt. > >> >> One thing that should be checked is if the new keytab and old keytab > >> >> have the same keys in them, and, if so just stop re-fetching and raising > >> >> a fatal error (at list for a reasonable period of time). > >> > > >> >I thought about a flag in the ipa_ad_server_ctx structure, because after > >> >the trust is deleted and set up again, the kvno is still 1. When you say > >> >'keys', are you proposing to compare the secret key? > >> >(krb5_keytab_entry.key) > >> Yes, don't use KVNO value to compare, verify actual keys. > > > >It would be nice to avoid the need to link directly to krb5 calls, but > >it is probably safer to check by key value. > Note that we are dealing with a key material from a separate realm and > we create the key in IPA LDAP by means of ipasam which results in the > fact there is no way to co-relate KVNO between the key in IPA LDAP and in AD > LDAP other than by authenticating or comparing key value to the local > copy. We need to be careful with this, IIRC there are some check on KVNO, we may have to force the KVNO to always be 0 (or 1 ?) to avoid the checks. > On the other hand, when trust is re-established, we actually remove it > completely and re-create the objects. This means for new trust KVNO will > always be 1 and will only increase if there was a TDO password change > operation initiated by AD side (which currently doesn't work, we have > this part broken). So I think we probably need to remember a timestamp > of when trust object -- not the Kerberos principal but the trust object > that ipasam maintains for Samba -- was created/modified and compare that > with the time we requested keytab before as a short circuit. As we run > on the same server as LDAP server, that time should be comparable. > > >> >> > - Should the IPA tools signal SSSD either via UNIX signals or via > >> >> > the DBus API once a trust has been added or removed to minimize > >> >> > the race condition and improve user experience? Currently I hope > >> >> > not, the proposed mechanism should be enough but testing is > >> >> > still > >> >> > required. > >> >> > > >> >> > When the design page is agreed on, I'll turn the three subsections of > >> >> > the implementation phase into smaller tickets and close the tracker. > >> >> > > >> >> > = Feature Name = > >> >> > > >> >> > Related ticket(s): > >> >> > * https://fedorahosted.org/sssd/ticket/2579 > >> >> > > >> >> > === Problem statement === > >> >> > The next IPA release will support one-way trusts. SSSD needs to add > >> >> > support > >> >> > for this feature in its server mode. > >> >> > > >> >> > === Use cases === > >> >> > One-way trust to Active Directory where FreeIPA realm trusts Active > >> >> > Directory > >> >> > forest using cross-forest trust feature of AD but the AD forest does > >> >> > not trust > >> >> > FreeIPA realm. Users from AD forest can access resources in FreeIPA > >> >> > realm. > >> >> > > >> >> > === Overview of the solution === > >> >> > At a high level, SSSD needs to examine the trust objects whether they > >> >> > are one-way or two way trusts. For each one-way trust, SSSD needs to > >> >> > fetch > >> >> > and store the keytab and use the keytab to secure the connection. For > >> >> > two-way trusts, we can keep using the existing code that reuses the > >> >> > IPA > >> >> > realm and the system keytab for both IPA and AD connectins. Care must > >> >> > be > >> >> > taken to remove keytabs of trusts that were removed as well. > >> >> > > >> >> > Fetching the keytab would be done by calling the `ipa-getkeytab` > >> >> > utility > >> >> > for every one-way trust. The keytab would only be (re)fetched if it's > >> >> > missing or if attempting to use the keytab failed. On the IPA server, > >> >> > we must make sure that the IPA server identity is allowed to read the > >> >> > keytab. > >> >> > > >> >> > Because handling multiple keytabs increases the risk of failing > >> >> > connections in > >> >> > case the trust wasn't setup correctly, we need to modify the failover > >> >> > code to > >> >> > not set the whole back end offline in case connecting to a subdomain > >> >> > AD server > >> >> > fails. Instead, the subdomain will be marked as inactive for a short > >> >> > period of > >> >> > time, during which it would act as offline. The proper way of solving > >> >> > this > >> >> > problem would be to rework the failover module so that it can act per > >> >> > domain, > >> >> > not only per back end, however, that change is out of scope for this > >> >> > release. > >> >> > > >> >> > === Implementation details === > >> >> > This section describes all the required changes in detail. > >> >> > > >> >> > ==== Reading the subdomains in the IPA subdomain handler ==== > >> >> > The IPA subdomain handler will include the attribute > >> >> > `ipaNTTrustDirection` > >> >> > when reading the trust objects. Currently this attribute is not > >> >> > readable > >> >> > by the host principal, so the IPA ACIs must be relaxed (ipa ticket?). > >> >> > If > >> >> > the trust direction is set to an OR of > >> >> > `lsa.LSA_TRUST_DIRECTION_OUTBOUND` > >> >> > and `lsa.LSA_TRUST_DIRECTION_INBOUND`, then it's a two-way trust and > >> >> > we'll > >> >> > just use the existing code that re-uses the IPA keytab for the AD > >> >> > trusted > >> >> > domain as well. If the attribute is only > >> >> > `lsa.LSA_TRUST_DIRECTION_OUTBOUND`, > >> >> > we handle the trust as a one-way trust. The trust type can be stored > >> >> > in > >> >> > `ipa_ad_server_ctx`. > >> >> > >> >> What do you do if you have only lsa.LSA_TRUST_DIRECTION_INBOUND ? > >> > > >> >Log this object loudly and ignore. Is there another reasonable thing to > >> >do other than skip such trust relationship ? (Yes, skipping the object > >> >should be spelled out explicitly in the design page) > >> Yes, nothing else you can do. I'm not sure at which point we'll get to > >> INBOUND-only case. > > > >We should probably just ignore w/o even screaming too loud. > Yep. > > > > >> >> > Each `sss_domain_info` structure will be created as `inactive` in the > >> >> > subdomain code. After enumerating the trusted domains, the subdomain > >> >> > handler will check if a keytab already exists for every one-way > >> >> > trusted > >> >> > domain. If yes, the domain is ready to use and can be enabled. If > >> >> > there is > >> >> > no keytab, the subdomain handler will fork out a call to > >> >> > `ipa-getkeytab`, > >> >> > fetch the keytab and store it under `/var/lib/sss/db`. The > >> >> > `ipa-getkeytab` > >> >> > call will be done using Kerberos credentials the host has. IPA ACIs > >> >> > must > >> >> > be modified accordingly to allow the IPA server principals to fetch > >> >> > the > >> >> > trust keytabs, but noone else. > >> >> > > >> >> > The directory `/var/lib/sss/db` is already only accessible to the sssd > >> >> > user only. As an additional security measure, we might want to store > >> >> > the > >> >> > keytabs under a new directory so that the keytabs also receive a > >> >> > SELinux > >> >> > context stricter than the default `sssd_var_lib_t`. If fetching the > >> >> > keytab > >> >> > succeeds, the domain would be enabled. The SELinux policy must also > >> >> > be > >> >> > adjusted to allow calling `ipa-getkeytab` by the `sssd_be` process. > >> >> > >> >> Getting a keytab is an LDAP extended operation, do you want to fork off > >> >> ipa-getkeytab to avoid coding the LDAP operatin in SSSD ? Or for some > >> >> other reason ? > >> > > >> >No, I just wanted to use code that is already there. Retrieving the > >> >keytab should be rare enough to justify the overhead of forking an > >> >executable and would let us to keep up with any changes IPA might do. > >> > > >> >Or let me turn that question around -- do you see any advantage of > >> >searching the keytab directly from SSSD? > >> I can see a downside in the fact that you'd need to handle ASN.1 > >> encoding/decoding to fetch the keytab and if we ever change its > >> structure, SSSD would be another point of failure. Calling out to > >> ipa-getkeytab is giving us assurance to keep things working. > > > >We cannot change the format, if we do we'll have to indtroduce a new > >version and support the old one. So this is not a problem. > > > >The advantage of doing this directly is if we decide (for whatever > >reason) to store the key material directly in the ldb cache instead of a > >file. If we did that then we could use a MEMORY keytab for the various > >operations by fetching it out of the cache and it would not ever hit the > >filesystem. Hoewever we may not want to have that key material in LDB at > >all, it is just a possibility. > Right, it is currently easier to call out and leave ipa-getkeytab to do > the job because we are practically running on IPA master in this more. Fine, I just wanted to know the reasons. > >> >> > If any trust relationships were removed, the corresponding keytabs > >> >> > must > >> >> > be removed from the disk as well. > >> >> > >> >> How do you do this ? > >> >> Just regularly enumerate the keytab files on disk and remove the ones > >> >> that have no object in the cache ? > >> > > >> >Yes, after the subdomain handler finishes. We alrady have a return boolean > >> >'refresh_has_changes' and re-initialize domain-realm mapping for > >> >example. > >> > > >> >> One advantage of using a separate subdir for keytabs, as you propose, > >> >> would be to wipe it clean every time sssd restart (at stop and at start > >> >> too for good measure), then enumerate the trusts on the server and > >> >> re-fetch any keytab. > >> > > >> >I thought about this, but it seemed a bit wasteful. But now that I think > >> >of it, this would present a 'known control point' to start fresh..so it > >> >might come handy. > >> If you do so, please make it triggered via some explicit mechanism, not > >> as a default. > > > >Why not do it at restart ? > Do you want to always re-retrive the keys? I was under assumption we are > not going to do that. Well we can have an option, or always do that, restarts are rare, esp on a server, so I thought it may be a simple way to start from a known clean slate. > >> >> > ==== Changes to the AD id_ctx instantiation ==== > >> >> > With two-way trust, we can keep using the default IPA principal and > >> >> > keytab. > >> >> > > >> >> > With one-way trust, the keytab retrieved from the IPA server must be > >> >> > used. Also, the principal must be passed into the > >> >> > `ad_create_default_options` > >> >> > function. The custom values must be set before we proceed to > >> >> > instantiate > >> >> > LDAP provider options. The only AD provider option we need to set is > >> >> > AD_KRB5_REALM. > >> >> > > >> >> > In the LDAP provider, we must take care that the following sdap > >> >> > options > >> >> > are set correctly: > >> >> > * SDAP_SASL_AUTHID - must be set to the NetBIOS name of the IPA > >> >> > domain. (A > >> >> > domain `TRUST.COM` would set this value to `TRUST$`). > >> >> > >> >> The NetBIOS domain can be completely unrelated to the DNS domain, so > >> >> make sure you do not just try to truncate the realm but actually fetch > >> >> the info from IPA/AD. > >> > > >> >Sure, sorry, that was meant as an example only. I added a more explicit > >> >sentence. > >> > > >> >> > >> >> > * SDAP_SASL_REALM - must be set to the AD realm > >> >> > * SDAP_KRB5_KEYTAB - must be set to the per-domain keytab > >> >> > retrieved from IPA > >> >> > > >> >> > The AD provider eventually calls `sdap_set_sasl_options()` from the > >> >> > LDAP provider, we need to make sure this function receives the correct > >> >> > values. During experimentation we were able to show that using > >> >> > multiple > >> >> > different SASL users and different realms doesn't cause any problems > >> >> > in > >> >> > SASL or LDAP libraries. > >> >> > > >> >> > The only place that will keep using the IPA realm is the failover > >> >> > instantiation. We need to keep using this hack until failover is > >> >> > per-backend. > >> >> > > >> >> > ==== Subdomain offline status changes ==== > >> >> > At the moment, the whole back end can be either online or offline and > >> >> > the status applies to both the main domain and the subdomains alike. > >> >> > As > >> >> > an effect, a failure to connect to a subdomain server would also make > >> >> > the > >> >> > main domain operate offline. In many subdomain setups, it's actually > >> >> > more > >> >> > convenient not to, because the subdomain server might be on a > >> >> > different > >> >> > network segment, behind a different firewall etc. Instead, the domain > >> >> > should only be made inactive. > >> >> > >> >> Does this cover accurately the Forest vs AD subdomain cases ? > >> >> We will have a keytab only for the AD root domain (the forest) and we > >> >> need to use those credentials for any other domain in the AD forest. > >> > > >> >Well, both subdomains would be made inactive, then. The point is to > >> >avoid marking the IPA domain as offline if the trusts break. > >> Yes, if you don't have keytab with keys to TDO object, you cannot > >> authenticate against any of the forest's AD DCs, in any domain of the > >> forest, so all of them would be offline in this case. Our current > >> problem is that we offline IPA domain here instead of the subdomains. > >> > >> > >> >btw this would be also useful for AD-AD trusts because what I learned is > >> >that AD subdomains tend to be isolated from one another on the network > >> >level as well and then failure to reach other domains is currently fatal > >> >which brings the whole sssd_be process offline. > >> > > >> >> > >> >> > The `sss_domain_info` structure would convert the 'bool disabled' > >> >> > parameter > >> >> > into an `enum sss_domain_state`. The supported values would be: > >> >> > * ''disabled'' - the domain should not be used by either > >> >> > responder or > >> >> > provider. It was removed or disabled on the server. > >> >> > * ''active'' - the domain can be used by a responder and the data > >> >> > provider > >> >> > would forward its request to the backend > >> >> > * ''inactive'' - the domain can be used by a responder, but the > >> >> > data > >> >> > provider would just shortcut as if the domain was offline. For > >> >> > now, this > >> >> > option will be used by subdomains only. > >> >> > > >> >> > The implementation would include renaming the existing > >> >> > `be_mark_offline()` > >> >> > function to be called `be_mark_dom_offline()` and modifying its > >> >> > behaviour. The > >> >> > existing code that sets the offline status and runs the offline > >> >> > callbacks > >> >> > would be called for parent domains only. For subdomains, we would > >> >> > mark the > >> >> > subdomain as inactive and schedule a tevent request that would > >> >> > unconditionally > >> >> > reset inactive domain to active. The request would be scheduled after > >> >> > `offline_timeout` seconds to be consistent with main domains from > >> >> > user's > >> >> > perspective. Likewise, the `be_reset_offline()` function will be > >> >> > extended to > >> >> > reset inactive domains to active as well as the SIGUSR1 and SIGUSR2 > >> >> > signal > >> >> > handlers. Finally, all calls to the `be_is_offline()` function should > >> >> > be > >> >> > inspected and the invocations that are per-domain should be converted > >> >> > to a new > >> >> > function `be_dom_is_offline()` that would check the offline status > >> >> > for parent > >> >> > domains and the offline state for subdomains. We should also make > >> >> > sure the > >> >> > backend offline status structure is opaque as currently its internals > >> >> > are > >> >> > readable by external users as well. Making the offline status opaque > >> >> > would > >> >> > make it safer to perform modifications to the offline code. > >> >> > > >> >> > In both offline and inactive cases, the ID handlers would reply with > >> >> > `DP_ERR_OFFLINE`. The crucial difference between offline and inactive > >> >> > at this > >> >> > point would be that inactive domains are re-activated > >> >> > undonditionally. When we > >> >> > modify the failover code to handle domains separately, we'll be able > >> >> > to > >> >> > leverage per-domain online checks or online/offline callbacks as well. > >> >> > > >> >> > ==== Detecting re-established trusts ==== > >> >> > In cases the `sdap_kinit_send()` request fails, the sdap code would > >> >> > return a > >> >> > special error code instead of blindly returning `EIO` as it does at > >> >> > the > >> >> > moment. When the `ipa_get_ad_acct` request receives this error code, > >> >> > it would > >> >> > re-run the subdomain request in order to check if the trust > >> >> > relationship still > >> >> > exists and in order to re-fetch the keytab again. In case the trust > >> >> > is still > >> >> > established, we would re-set the inactive domain status and re-run > >> >> > the account > >> >> > request. In case the trust went away, the subdomain code should > >> >> > remove the > >> >> > trusted domain (however, this must be tested). If the account request > >> >> > returns > >> >> > any error after reinitializing the ketab, the subdomain would be > >> >> > marked > >> >> > inactive in order to avoid fetching the keytab again and again. In > >> >> > order to be > >> >> > able to run the subdomain request separately from the subdomain back > >> >> > end > >> >> > handler, the subdomain code must be wrapped into a subdomain request > >> >> > as the > >> >> > code currently assumes it's being called from the subdomain backend > >> >> > handler > >> >> > only. > >> >> > > >> >> > === Future work === > >> >> > Handling failover and offline status on per-domain basis instead of > >> >> > per-backend basis should be done in the next release. > >> >> > > >> >> > === Configuration changes === > >> >> > none > >> >> > > >> >> > === How To Test === > >> >> > Establish a one-way trust relationship with an AD domain. Make sure > >> >> > both > >> >> > IPA and AD users are resolvable. It's prudent to test combinations of > >> >> > one-way and two-way trusts with different forests. Make sure removing > >> >> > a > >> >> > trust relationship removes the keytab from the filesystem. Make sure > >> >> > that > >> >> > SSSD handles re-establishing a trust relationship. > >> >> > > >> >> > === Authors === > >> >> > * Jakub Hrozek <jhro...@redhat.com> > >> >> > >> >> Good job. > >> > > >> >Thank you for taking the time to review the design. > >> > > >> >I made some changes to the design page, here is the diff: > >> >https://fedorahosted.org/sssd/wiki/DesignDocs/OneWayTrusts?action=diff&version=9&old_version=8 > >> > > >> >I'll do one more change after we agree on how should we protect against > >> >loops in fetching keytabs (flag vs. check keytab contents). > >> I think the design is sensible. > > > >Ack. > > > >Simo. > > > _______________________________________________ sssd-devel mailing list sssd-devel@lists.fedorahosted.org https://lists.fedorahosted.org/mailman/listinfo/sssd-devel