Re: [Freeipa-users] NetworkError : invalid continuation byte with utf8 codec

[Fraser Tweedale]

On Tue, Dec 22, 2015 at 08:39:09AM +0100, Gmail wrote:
> Here are the files you ask for:
> 
Thank you.  I see Tomcat is running in an fr_FR locale. Could you
also provide contents of `/etc/locale.conf'?

Cheers,
Fraser

> 
> 
> Le 22 décembre 2015 à 02:30:06, Fraser Tweedale (ftwee...@redhat.com) a écrit:
> 
> On Mon, Dec 21, 2015 at 05:29:01PM +0100, Gmail wrote:  
> > Hi all,  
> >  
> > When trying to install on a fresh new Centos 7 I’ve got this error :  
> >  
> > 2015-12-21T16:04:44Z DEBUG The ipa-server-install command failed, 
> > exception: NetworkError: cannot connect to 
> > 'https://freeipa.ipa.local:8443/ca/rest/profiles/raw': 'utf8' codec can't 
> > decode byte 0xea in position 13: invalid continuation byte  
> > 2015-12-21T16:04:44Z ERROR cannot connect to 
> > 'https://freeipa.ipa.local:8443/ca/rest/profiles/raw': 'utf8' codec can't 
> > decode byte 0xea in position 13: invalid continuation byte  
> >  
> > My freeipa-server version is :  4.2.0  
> > I’m running a Centos 3.10.0-327.3.1.el7.x86_64  
> >  
> > Any idea of what goes wrong?  
> >  
> Thanks for reporting. I have not seen this error before. Could you  
> please include the following log files and I will take a closer  
> look:  
> 
> /var/log/ipaserver-install.log  
> /var/log/pki/pki-tomcat/ca/debug  
> 
> Cheers,  
> Fraser  



-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

Re: [Freeipa-users] Two Factor = SSHKeys + OTP or Password

[Sumit Bose]

On Tue, Dec 22, 2015 at 06:51:25PM +0530, Yogesh Sharma wrote:
> Hi List,
> 
> Did not see any options for SSH Keys + OTP or Password, However would like
> to know if it is possible with FreeIPA user.
> 
> With Generic SSH , We can use use AuthenticationMethods, but not sure where
> to check in FreeIPA.

I think there is nothing specific about FreeIPA here. If you set on a
IPA client 'AuthenticationMethods = publickey,password' in sshd_config,
sshd will check the ssh key first and then ask the user for a password.

If the user is configured to use OTP on the IPA server then you have to
enter not only the password but the OTP token as well.

HTH

bye,
Sumit

> 
> 
> 
> 
> *Best Regards,*
> 
> *__*
> 
> *Yogesh Sharma*
> *Email: yks0...@gmail.com  | Web: www.initd.in
>  *
> 
> *RHCE, VCE-CIA, RACKSPACE CLOUD U Certified*
> 
>    
> 
> 

> -- 
> Manage your subscription for the Freeipa-users mailing list:
> https://www.redhat.com/mailman/listinfo/freeipa-users
> Go to http://freeipa.org for more info on the project

-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

Re: [Freeipa-users] Want faster user-add

[Simo Sorce]

On Tue, 2015-12-22 at 10:24 +0100, thierry bordaz wrote:
> On 12/21/2015 05:55 PM, Daryl Fonseca-Holt wrote:
> > Hi all,
> >
> > Environment: RHEL6 with IPA 3.0 at current RedHat level. 64-core 
> > 256-GB RAM Oracle x4470 M2.
> >
> > During our migration from NIS on Solaris 140,000+ accounts will be 
> > added. After tuning per the guides dbmon.sh shows no roevicts and we 
> > get high cache hit ratios.
> >
> > Per a previous discussion with the list the input is broken down into 
> > batches of less than 1,000 users and the default IPA group is changed 
> > before each batch. This helped greatly.
> >
> > Adding all the users takes many hours. Initially ipa user-add takes an 
> > average 2.3 seconds per user but degrades by the time there are 
> > 140,000 users to an average 6.7 seconds per user.
> >
> > In tracing it appears that a significant portion of the time ipa 
> > user-add takes is not the add itself, it is the query at the end that 
> > displays the resulting user account. Is there any legit way to prevent 
> > this query?
> >
> > The length of time it takes to migrate is not a big concern. The 
> > concern is the start of the fall school term when we typically add 
> > approximately 1,300 accounts per hour during the registration period 
> > with our current system.
> >
> > All suggestions will be appreciated.
> >
> > Regards, Daryl
> >
> Hi Daryl,
> 
> I can reproduce similar trend of user-add becoming slower and slower.
> 
> Now in my tests (etime=7s) the time was spent half by authentication and 
> half by ADD and MOD (update of ipausers group). I agree there are many 
> direct SRCH (~10) but they all seems to be rapid.
> 
> I know that the vast majority of the time is spent in DS schema-compat 
> plugin. Disabling it, during provisioning, reduce the duration by ~3.
> Now I do not know if it is a valid option to disable this plugin during 
> provisioning.

As long as the schema compat is not needed by users during the
provisioning, turning it off is fine. All the contents are regenerated
at startup anyway. So it can be re-enabled and the server restarted
after the bulk provisioning is done.

Simo.

-- 
Simo Sorce * Red Hat, Inc * New York

-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

[Freeipa-users] Two Factor = SSHKeys + OTP or Password

[Yogesh Sharma]

Hi List,

Did not see any options for SSH Keys + OTP or Password, However would like
to know if it is possible with FreeIPA user.

With Generic SSH , We can use use AuthenticationMethods, but not sure where
to check in FreeIPA.




*Best Regards,*

*__*

*Yogesh Sharma*
*Email: yks0...@gmail.com  | Web: www.initd.in
 *

*RHCE, VCE-CIA, RACKSPACE CLOUD U Certified*

   


-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

Re: [Freeipa-users] Want faster user-add

[Daryl Fonseca-Holt]

On 12/22/15 08:09, Petr Vobornik wrote:

On 12/22/2015 10:24 AM, thierry bordaz wrote:

On 12/21/2015 05:55 PM, Daryl Fonseca-Holt wrote:

Hi all,

Environment: RHEL6 with IPA 3.0 at current RedHat level. 64-core
256-GB RAM Oracle x4470 M2.

During our migration from NIS on Solaris 140,000+ accounts will be
added. After tuning per the guides dbmon.sh shows no roevicts and we
get high cache hit ratios.

Per a previous discussion with the list the input is broken down into
batches of less than 1,000 users and the default IPA group is changed
before each batch. This helped greatly.

Adding all the users takes many hours. Initially ipa user-add takes an
average 2.3 seconds per user but degrades by the time there are
140,000 users to an average 6.7 seconds per user.

In tracing it appears that a significant portion of the time ipa
user-add takes is not the add itself, it is the query at the end that
displays the resulting user account. Is there any legit way to prevent
this query?

The length of time it takes to migrate is not a big concern. The
concern is the start of the fall school term when we typically add
approximately 1,300 accounts per hour during the registration period
with our current system.

All suggestions will be appreciated.

Regards, Daryl


Hi Daryl,

I can reproduce similar trend of user-add becoming slower and slower.

Now in my tests (etime=7s) the time was spent half by authentication and
half by ADD and MOD (update of ipausers group). I agree there are many
direct SRCH (~10) but they all seems to be rapid.

I know that the vast majority of the time is spent in DS schema-compat
plugin. Disabling it, during provisioning, reduce the duration by ~3.
Now I do not know if it is a valid option to disable this plugin during
provisioning.

thanks
thierry


We must also distinguish performance on IPA 3.x (RHEL 6.x) and FreeIPA 
4.2/4.3


FreeIPA 4.2 got some performance improvements mostly related to group 
membership handling.


Improving user-add is one of primary goals for 4.4 release:
* https://fedorahosted.org/freeipa/ticket/5448

There are couple issues tracked about plugins output (also planned to 
be fixed in 4.4):

* https://fedorahosted.org/freeipa/ticket/5281
* https://fedorahosted.org/freeipa/ticket/5282
* https://fedorahosted.org/freeipa/ticket/4995

You can try to call user-add with --raw options but it won't help much 
because some parts ignore it. Other than that, there is not clean 
workaround.


When user is added, the user_add typically:
* adds the user to ipadefaultprimarygroup
* converts manger dn to human friedly value (should not cause perf. 
issues)
* set description to magic value to cause generation of user private 
group (calls user-mod)
* fetches password attributes and resolves ssh pub keys (does couple 
of searches)


An unsupported possibility - do on your own risk - is to remove these 
operations from user_add post_callback in ipalib/plugins/user.py 
(around line 535 on 6.7).


Other thing which might help is to remove 'memberof' and 
'memberofindirect' values from default_attributes in user.py (~line 
216). Note that it also affects other user-* commands.


All should be tried in testing environment.

Another performance improvement is to call IPA API directly and use 
batch command - with that it is possible to add e.g. 100 users with 
one call and save some network calls.


Example could be seen in this ugly script: 
https://pvoborni.fedorapeople.org/scripts/ipa-generate-users.sh
I did test with RHEL7 and IPA 4.2 and the performance was much better. 
We don't have RHEL7 deployed yet so there are some issues with change 
management but we may have to revisit that.


I looked at changing the code but we can't end up with an unsupported 
product. We depend on our Red Hat support heavily for production. So I 
could speed up the migration from the NIS server but I won't be able to 
keep the optimization during day-to-day operation.


Thanks for the script. Ugly scripts are good learning examples because 
it's easy to see the real purpose without the distraction of frills.


Regards, Daryl

--
 --
 Daryl Fonseca-Holt
 IST/CNS/Unix Server Team
 University of Manitoba
 204.480.1079

--
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

Re: [Freeipa-users] Want faster user-add

[Daryl Fonseca-Holt]

On 12/22/15 03:24, thierry bordaz wrote:

On 12/21/2015 05:55 PM, Daryl Fonseca-Holt wrote:

Hi all,

Environment: RHEL6 with IPA 3.0 at current RedHat level. 64-core 
256-GB RAM Oracle x4470 M2.


During our migration from NIS on Solaris 140,000+ accounts will be 
added. After tuning per the guides dbmon.sh shows no roevicts and we 
get high cache hit ratios.


Per a previous discussion with the list the input is broken down into 
batches of less than 1,000 users and the default IPA group is changed 
before each batch. This helped greatly.


Adding all the users takes many hours. Initially ipa user-add takes 
an average 2.3 seconds per user but degrades by the time there are 
140,000 users to an average 6.7 seconds per user.


In tracing it appears that a significant portion of the time ipa 
user-add takes is not the add itself, it is the query at the end that 
displays the resulting user account. Is there any legit way to 
prevent this query?


The length of time it takes to migrate is not a big concern. The 
concern is the start of the fall school term when we typically add 
approximately 1,300 accounts per hour during the registration period 
with our current system.


All suggestions will be appreciated.

Regards, Daryl


Hi Daryl,

I can reproduce similar trend of user-add becoming slower and slower.

Now in my tests (etime=7s) the time was spent half by authentication 
and half by ADD and MOD (update of ipausers group). I agree there are 
many direct SRCH (~10) but they all seems to be rapid.


I know that the vast majority of the time is spent in DS schema-compat 
plugin. Disabling it, during provisioning, reduce the duration by ~3.
Now I do not know if it is a valid option to disable this plugin 
during provisioning.


thanks
thierry
Thanks for validating my timings, it's good to know I'm not off track 
somewhere.


Not being sure what I will end up with if the DS schema-compat plugin is 
disabled I'm not sure I'll end up with what we need. We need NIS in the 
future and further out we will convert the clients to LDAP and possible 
SSSD in the case of Linux clients.


Thanks, Daryl

--
 --
 Daryl Fonseca-Holt
 IST/CNS/Unix Server Team
 University of Manitoba
 204.480.1079

-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

Re: [Freeipa-users] Want faster user-add

[Petr Vobornik]

On 12/22/2015 10:24 AM, thierry bordaz wrote:

On 12/21/2015 05:55 PM, Daryl Fonseca-Holt wrote:

Hi all,

Environment: RHEL6 with IPA 3.0 at current RedHat level. 64-core
256-GB RAM Oracle x4470 M2.

During our migration from NIS on Solaris 140,000+ accounts will be
added. After tuning per the guides dbmon.sh shows no roevicts and we
get high cache hit ratios.

Per a previous discussion with the list the input is broken down into
batches of less than 1,000 users and the default IPA group is changed
before each batch. This helped greatly.

Adding all the users takes many hours. Initially ipa user-add takes an
average 2.3 seconds per user but degrades by the time there are
140,000 users to an average 6.7 seconds per user.

In tracing it appears that a significant portion of the time ipa
user-add takes is not the add itself, it is the query at the end that
displays the resulting user account. Is there any legit way to prevent
this query?

The length of time it takes to migrate is not a big concern. The
concern is the start of the fall school term when we typically add
approximately 1,300 accounts per hour during the registration period
with our current system.

All suggestions will be appreciated.

Regards, Daryl


Hi Daryl,

I can reproduce similar trend of user-add becoming slower and slower.

Now in my tests (etime=7s) the time was spent half by authentication and
half by ADD and MOD (update of ipausers group). I agree there are many
direct SRCH (~10) but they all seems to be rapid.

I know that the vast majority of the time is spent in DS schema-compat
plugin. Disabling it, during provisioning, reduce the duration by ~3.
Now I do not know if it is a valid option to disable this plugin during
provisioning.

thanks
thierry


We must also distinguish performance on IPA 3.x (RHEL 6.x) and FreeIPA 
4.2/4.3


FreeIPA 4.2 got some performance improvements mostly related to group 
membership handling.


Improving user-add is one of primary goals for 4.4 release:
* https://fedorahosted.org/freeipa/ticket/5448

There are couple issues tracked about plugins output (also planned to be 
fixed in 4.4):

* https://fedorahosted.org/freeipa/ticket/5281
* https://fedorahosted.org/freeipa/ticket/5282
* https://fedorahosted.org/freeipa/ticket/4995

You can try to call user-add with --raw options but it won't help much 
because some parts ignore it. Other than that, there is not clean 
workaround.


When user is added, the user_add typically:
* adds the user to ipadefaultprimarygroup
* converts manger dn to human friedly value (should not cause perf. issues)
* set description to magic value to cause generation of user private 
group (calls user-mod)
* fetches password attributes and resolves ssh pub keys (does couple of 
searches)


An unsupported possibility - do on your own risk - is to remove these 
operations from user_add post_callback in ipalib/plugins/user.py (around 
line 535 on 6.7).


Other thing which might help is to remove 'memberof' and 
'memberofindirect' values from default_attributes in user.py (~line 
216). Note that it also affects other user-* commands.


All should be tried in testing environment.

Another performance improvement is to call IPA API directly and use 
batch command - with that it is possible to add e.g. 100 users with one 
call and save some network calls.


Example could be seen in this ugly script: 
https://pvoborni.fedorapeople.org/scripts/ipa-generate-users.sh

--
Petr Vobornik

--
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

Re: [Freeipa-users] Purge old entries in /var/lib/dirsrv/slapd-xxx/cldb/xxx.db4 file

[Ludwig Krispenz]

Hi,

On 12/22/2015 11:43 AM, David Goudet wrote:

Hi,

I have multimaster replication environment. On each replica, folder 
/var/lib/dirsrv/slapd-/cldb/ has big size (3~GB) and old entries in 
/var/lib/dirsrv/slapd-xxx/cldb/xxx.db4 have three month year old:

sudo dbscan -f 
/var/lib/dirsrv/slapd-/cldb/ef155b03-dda611e2-a156db20-90xxx06_51c9aed900xx000.db4
 | less
dbid: 56239e5e0004
 replgen: 1445174777 Sun Oct 18 15:26:17 2015
 csn: 56239e5e0004
 uniqueid: e55d5e01-26f211e4-9b60db20-90c3b706
 dn: 
 operation: modify
 krbLastSuccessfulAuth: 20151018132617Z
 modifiersname: cn=Directory Manager
 modifytimestamp: 20151018132617Z
 entryusn: 68030946

My questions are:

a) How to purge old entries in file /var/lib/dirsrv/slapd-xxx/cldb/xxx.db4? 
(what is the procedure)
b) What is the right configuration to limit increase of this file?
setting changelog maxage should be sufficient to trim changes, but the 
age is not the only condition deciding if a recored in the changelog can 
be deleted.
- for each replicaID the last record will never be deleted, independent 
of its age, so if you have replicas in your topology which are not (or 
not frequently) updated directly there will be old changes in the changelog
- if the replica where the trimming is run and if it has replication 
agreements to other replicas, changes which were not yet replicated to 
the other replica will not be purged. So, if you have some stale 
agreements to other replicas this could prevent trimming as well.


Also trimming removes changelog records and frees space internally ro th 
edb4 file  to be reused, but it will not shrink the file size




This topic has been already talk on 
https://www.redhat.com/archives/freeipa-users/2013-February/msg00433.html or 
https://www.redhat.com/archives/freeipa-users/2015-April/msg00573.html but no 
response work for me.
Response here seems to be not applicable 
https://bugzilla.redhat.com/show_bug.cgi?id=1181341 (Centos 7, Fixed In 
Version: 389-ds-base-1.3.4.0-1.el7)

I used some attributes from the docuementation: 
https://access.redhat.com/documentation/en-US/Red_Hat_Directory_Server/9.0/html/Configuration_Command_and_File_Reference/Core_Server_Configuration_Reference.html#cnchangelog5-nsslapd_changelogdir.
 Old entries are not purged and file increase even after restart service 
(service dirvsrv start and service dirvsrv stop).

(This test environment values)
dn: cn=changelog5,cn=config
objectClass: top
objectClass: extensibleobject
cn: changelog5
...
nsslapd-changelogmaxentries: 100
nsslapd-changelogmaxage: 4m

dn: cn=replica,cn=x,cn=mapping tree,cn=config
cn: replica
nsDS5Flags: 1
objectClass: top
objectClass: nsds5replica
objectClass: extensibleobject
nsDS5ReplicaType: 3
nsDS5ReplicaRoot: dc=x
nsds5ReplicaLegacyConsumer: off
nsDS5ReplicaId: 6
nsDS5ReplicaBindDN: cn=replication manager,cn=config
nsDS5ReplicaBindDN: krbprincipalname=ldap/xx
  .LYRA,cn=services,cn=accounts,dc=x
nsState:: x
nsDS5ReplicaName: d9663d08-a80f11e5-aa48d241-0b88f012
nsds5ReplicaTombstonePurgeInterval: 200
nsds5ReplicaPurgeDelay: 200
nsds5ReplicaChangeCount: 3091
nsds5replicareapactive: 0

Hereafter some informations about my environment:
CentOS release 6.5 (Final)
389-ds-base-libs-1.2.11.15-65.el6_7.x86_64
389-ds-base-1.2.11.15-65.el6_7.x86_64
ipa-client-3.0.0-47.el6.centos.1.x86_64
ipa-server-3.0.0-47.el6.centos.1.x86_64

Thanks for your help!

David



--
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

[Freeipa-users] Queries on migrating nis netgroups

[Roderick Johnstone]

Hi

I'm migrating our nis environment to freeipa 4.2.0 on Redhat 7.

I need to have the netgroups set up in freeipa before migrating systems 
to be freeipa clients.


At this point I'm trying to understand the relationship between 
hostgroups and netgroups and whether I should just be using ipa 
netgroup-add and ipa netgroup-add-member commands or whether I should be 
using equivalent ipa hostgroup* commands.


Section 14.5.1 of the Redhat 7 Domain Identity Authentication and Policy 
Guide is telling me that I get a shadow netgroup for every hostgroup I 
create and that I can manage these netgroups with the 
"ipa-host-net-manage" command.


I don't see the ipa-host-net-manage command. There are
ipa host* commands but these don't include ipa host-net* commands. What 
am I missing here?


Also the ipa netgroup* commands don't seem to be able to manage the 
shadow netgroups so I'm currently unable to manipulate my shadow 
netgroups to eg change the nisdomain associated with them. How do I do that?


Also it looks like I can't add non-ipa clients into hostgroups so 
presumable not into shadow netgroups either, so maybe this is a 
non-starter for me. Did I understand that correctly?


Thanks

Roderick Johnstone

--
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

Re: [Freeipa-users] ipa-replica-prepare error: Profile caIPAserviceCert Not Found

[Fraser Tweedale]

On Tue, Dec 22, 2015 at 10:06:55AM +0100, Karl Forner wrote:
> Hi Fraser,
> The ipa-replica-prepare ran in a adelton/freeipa-server:lastest-systemd
> docker, which I think is based on fedora 23 and contains freeIPA v 4.2.3.
> I can try to patch it, but I'm really not used to fedora, and moreover
> there's a debian/docker bug that prevents me from building the docker image
> on my computers.
> 
> Thanks,
> Karl
> 
OK, fair enough.  A couple of follow-up questions:

- Is the issue always reproducible or only some of the time?

- Are you running replica-prepare immediately after starting the
  container?  Does the issue still occur after waiting a while?

If you attach your /var/log/pki/pki-tomcat/ca/debug log it will help
pinpoint the cause and confirm/deny whether the existing patch will
fix it.

Cheers,
Fraser

> On Tue, Dec 22, 2015 at 2:46 AM, Fraser Tweedale 
> wrote:
> 
> > On Mon, Dec 21, 2015 at 01:57:02PM +0100, Karl Forner wrote:
> > > Hello,
> > >
> > > Running:
> > > ipa-replica-prepare ipa-h3s1.example.com --ip-address xx.xx.xx.xx -d -v
> > > fails
> > > with
> > > ipa: DEBUG: Protocol: TLS1.2
> > > ipa: DEBUG: Cipher: TLS_RSA_WITH_AES_128_CBC_SHA
> > > ipa: DEBUG: request status 200
> > > ipa: DEBUG: request reason_phrase u'OK'
> > > ipa: DEBUG: request headers {'date': 'Mon, 21 Dec 2015 12:50:59 GMT',
> > > 'content-length': '148', 'content-type': 'application/xml', 'server':
> > > 'Apache-Coyote/1.1'}
> > > ipa: DEBUG: request body ' > > standalone="no"?>1Profile
> > > caIPAserviceCert Not Found'
> > > ipa.ipaserver.install.ipa_replica_prepare.ReplicaPrepare: DEBUG:   File
> > > "/usr/lib/python2.7/site-packages/ipapython/admintool.py", line 171, in
> > > execute
> > >
> > > The context is probably unusual:
> > > I run the command on a replica with CA from a server in freeipa v4.1.4
> > (in
> > > a adelton/freeipa-server docker)
> > > which is a freeipa v4.2.3  running in
> > > adelton/freeipa-server:lastest-systemd docker
> > >
> > > I found this ticket which looks similar:
> > > https://fedorahosted.org/freeipa/ticket/5376
> > >
> > > Is there something wrong with my replica knowing that it has been
> > > replicated from a 4.1.4 ?
> > > Is there a work-around ?
> > >
> > > Thanks
> > > Karl
> >
> > Hi Karl,
> >
> > I have a patch for Dogtag that I think will fix this issue.  Would
> > you be willing to test it?  If so, which version of Fedora/RHEL are
> > you using and I will prepare a build.
> >
> > Regards,
> > Fraser
> >
> > > --
> > > Manage your subscription for the Freeipa-users mailing list:
> > > https://www.redhat.com/mailman/listinfo/freeipa-users
> > > Go to http://freeipa.org for more info on the project
> >
> >

-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

[Freeipa-users] Purge old entries in /var/lib/dirsrv/slapd-xxx/cldb/xxx.db4 file

[David Goudet]

Hi,

I have multimaster replication environment. On each replica, folder 
/var/lib/dirsrv/slapd-/cldb/ has big size (3~GB) and old entries in 
/var/lib/dirsrv/slapd-xxx/cldb/xxx.db4 have three month year old:

sudo dbscan -f 
/var/lib/dirsrv/slapd-/cldb/ef155b03-dda611e2-a156db20-90xxx06_51c9aed900xx000.db4
 | less
dbid: 56239e5e0004
replgen: 1445174777 Sun Oct 18 15:26:17 2015
csn: 56239e5e0004
uniqueid: e55d5e01-26f211e4-9b60db20-90c3b706
dn: 
operation: modify
krbLastSuccessfulAuth: 20151018132617Z
modifiersname: cn=Directory Manager
modifytimestamp: 20151018132617Z
entryusn: 68030946

My questions are:

a) How to purge old entries in file /var/lib/dirsrv/slapd-xxx/cldb/xxx.db4? 
(what is the procedure)
b) What is the right configuration to limit increase of this file?



This topic has been already talk on 
https://www.redhat.com/archives/freeipa-users/2013-February/msg00433.html or 
https://www.redhat.com/archives/freeipa-users/2015-April/msg00573.html but no 
response work for me.
Response here seems to be not applicable 
https://bugzilla.redhat.com/show_bug.cgi?id=1181341 (Centos 7, Fixed In 
Version: 389-ds-base-1.3.4.0-1.el7)

I used some attributes from the docuementation: 
https://access.redhat.com/documentation/en-US/Red_Hat_Directory_Server/9.0/html/Configuration_Command_and_File_Reference/Core_Server_Configuration_Reference.html#cnchangelog5-nsslapd_changelogdir.
 Old entries are not purged and file increase even after restart service 
(service dirvsrv start and service dirvsrv stop).

(This test environment values)
dn: cn=changelog5,cn=config
objectClass: top
objectClass: extensibleobject
cn: changelog5
...
nsslapd-changelogmaxentries: 100
nsslapd-changelogmaxage: 4m

dn: cn=replica,cn=x,cn=mapping tree,cn=config
cn: replica
nsDS5Flags: 1
objectClass: top
objectClass: nsds5replica
objectClass: extensibleobject
nsDS5ReplicaType: 3
nsDS5ReplicaRoot: dc=x
nsds5ReplicaLegacyConsumer: off
nsDS5ReplicaId: 6
nsDS5ReplicaBindDN: cn=replication manager,cn=config
nsDS5ReplicaBindDN: krbprincipalname=ldap/xx
 .LYRA,cn=services,cn=accounts,dc=x
nsState:: x
nsDS5ReplicaName: d9663d08-a80f11e5-aa48d241-0b88f012
nsds5ReplicaTombstonePurgeInterval: 200
nsds5ReplicaPurgeDelay: 200
nsds5ReplicaChangeCount: 3091
nsds5replicareapactive: 0

Hereafter some informations about my environment: 
CentOS release 6.5 (Final)
389-ds-base-libs-1.2.11.15-65.el6_7.x86_64
389-ds-base-1.2.11.15-65.el6_7.x86_64
ipa-client-3.0.0-47.el6.centos.1.x86_64
ipa-server-3.0.0-47.el6.centos.1.x86_64

Thanks for your help!

David

-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

Re: [Freeipa-users] ipa-replica-prepare error: Profile caIPAserviceCert Not Found

[Karl Forner]

Hi Fraser,
The ipa-replica-prepare ran in a adelton/freeipa-server:lastest-systemd
docker, which I think is based on fedora 23 and contains freeIPA v 4.2.3.
I can try to patch it, but I'm really not used to fedora, and moreover
there's a debian/docker bug that prevents me from building the docker image
on my computers.

Thanks,
Karl

On Tue, Dec 22, 2015 at 2:46 AM, Fraser Tweedale 
wrote:

> On Mon, Dec 21, 2015 at 01:57:02PM +0100, Karl Forner wrote:
> > Hello,
> >
> > Running:
> > ipa-replica-prepare ipa-h3s1.example.com --ip-address xx.xx.xx.xx -d -v
> > fails
> > with
> > ipa: DEBUG: Protocol: TLS1.2
> > ipa: DEBUG: Cipher: TLS_RSA_WITH_AES_128_CBC_SHA
> > ipa: DEBUG: request status 200
> > ipa: DEBUG: request reason_phrase u'OK'
> > ipa: DEBUG: request headers {'date': 'Mon, 21 Dec 2015 12:50:59 GMT',
> > 'content-length': '148', 'content-type': 'application/xml', 'server':
> > 'Apache-Coyote/1.1'}
> > ipa: DEBUG: request body ' > standalone="no"?>1Profile
> > caIPAserviceCert Not Found'
> > ipa.ipaserver.install.ipa_replica_prepare.ReplicaPrepare: DEBUG:   File
> > "/usr/lib/python2.7/site-packages/ipapython/admintool.py", line 171, in
> > execute
> >
> > The context is probably unusual:
> > I run the command on a replica with CA from a server in freeipa v4.1.4
> (in
> > a adelton/freeipa-server docker)
> > which is a freeipa v4.2.3  running in
> > adelton/freeipa-server:lastest-systemd docker
> >
> > I found this ticket which looks similar:
> > https://fedorahosted.org/freeipa/ticket/5376
> >
> > Is there something wrong with my replica knowing that it has been
> > replicated from a 4.1.4 ?
> > Is there a work-around ?
> >
> > Thanks
> > Karl
>
> Hi Karl,
>
> I have a patch for Dogtag that I think will fix this issue.  Would
> you be willing to test it?  If so, which version of Fedora/RHEL are
> you using and I will prepare a build.
>
> Regards,
> Fraser
>
> > --
> > Manage your subscription for the Freeipa-users mailing list:
> > https://www.redhat.com/mailman/listinfo/freeipa-users
> > Go to http://freeipa.org for more info on the project
>
>
-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

Re: [Freeipa-users] Want faster user-add

[thierry bordaz]

On 12/21/2015 05:55 PM, Daryl Fonseca-Holt wrote:

Hi all,

Environment: RHEL6 with IPA 3.0 at current RedHat level. 64-core 
256-GB RAM Oracle x4470 M2.


During our migration from NIS on Solaris 140,000+ accounts will be 
added. After tuning per the guides dbmon.sh shows no roevicts and we 
get high cache hit ratios.


Per a previous discussion with the list the input is broken down into 
batches of less than 1,000 users and the default IPA group is changed 
before each batch. This helped greatly.


Adding all the users takes many hours. Initially ipa user-add takes an 
average 2.3 seconds per user but degrades by the time there are 
140,000 users to an average 6.7 seconds per user.


In tracing it appears that a significant portion of the time ipa 
user-add takes is not the add itself, it is the query at the end that 
displays the resulting user account. Is there any legit way to prevent 
this query?


The length of time it takes to migrate is not a big concern. The 
concern is the start of the fall school term when we typically add 
approximately 1,300 accounts per hour during the registration period 
with our current system.


All suggestions will be appreciated.

Regards, Daryl


Hi Daryl,

I can reproduce similar trend of user-add becoming slower and slower.

Now in my tests (etime=7s) the time was spent half by authentication and 
half by ADD and MOD (update of ipausers group). I agree there are many 
direct SRCH (~10) but they all seems to be rapid.


I know that the vast majority of the time is spent in DS schema-compat 
plugin. Disabling it, during provisioning, reduce the duration by ~3.
Now I do not know if it is a valid option to disable this plugin during 
provisioning.


thanks
thierry
-- 
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

Re: [Freeipa-users] Issues with 'A replication agreement for the host already exists', when it very much doesn't

[Ludwig Krispenz]

On 12/21/2015 05:49 PM, Alex Williams wrote:
I began installing a new ipa4 replica this morning and it all went 
wrong. The ipa-replica-install script got all the way to restarting 
ipa with systemctl at the very end, having set up replication and then 
fell over, because systemctl couldn't find the ipa service. I removed 
the replica from our master, I deleted the host from there too, I 
un-installed ipa-server on the new replica machine, I even created a 
new replica-prepare script on the master, but now the server just 
errors immediately with:


A replication agreement for this host already exists. It needs to 
be removed.


I've verified several times, that no replica, or host with the same 
name exists in the master, there are no ldap entries under masters, 
with that hostname, nothing. There is literally no trace of the new 
host, on the old master. Running `ipa-replica-manage list` shows just 
the 3 ipa servers we have already, no sign of this new host. Yet, if I 
run `ipa-replica-manage del hostname --force` on the master, it will 
in fact say that it's forcing removal, skipping checking if anything 
will be orphaned and that no RUV records were found.


I'm now lost, I really don't know where to start with fixing this.
we should first try to get a clear picture of existing agreements and 
state of replication. Could you on all servers do the following searches 
(as directory manager)


ldapsearch -LLL -o ldif-wrap=no  . -b "cn=config" 
"objectclass=nsds5replicationagreement" nsDS5ReplicaRoot nsDS5ReplicaHost
ldapsearch -LLL -o ldif-wrap=no .. -b "cn=config" 
"objectclass=nsds5replica" nsDS5ReplicaRoot nsDS5ReplicaId nsds50ruv


Not sure if this is relevant or not, but I'd rather bring it up and it 
not be, than not mention it and it turn out to be the reason. Our yum 
mirror is unfortunately now holding rhel7.2 packages, whilst our 
servers, are still on rhel7.1, which means our existing IPA servers, 
are ipa4.1 and the new one I tried to install, was ipa4.2, but on a 
rhel7.1 box. I had previously attributed the failed systemctl command, 
to the fact that I was trying to run ipa4.2 on a rhel7.1 box, as I'm 
told there were a lot of modifications to systemctl in rhel7.2, but I 
need to fix this replication agreement issue, before I can try again 
with the box upgraded to rhel7.2.


Any ideas?

Cheers

Alex



--
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

Re: [Freeipa-users] OS X Yosemite unable to authenticate

[John Obaterspok]

Hi,

Are you only having problems to login to login to OSX with the IPA user
now? If that is the case then check the DNS settings you are using and make
sure the IPA server is listed first and that it has full name. Exactly the
same problem occurred for me with the slow logins to OSX which was due to
the DNS settings and that OSX only used short name of IPA server during
login (if I logged in as local user I could ping and lookup hosts using
short name)

-- john

2015-12-21 17:49 GMT+01:00 Nicola Canepa :

> I had to configure /etc/krb5.conf, and to avoid the requested reboot, I
> did a "dscacheutil -flushcache", both as the logged in user and as root.
> I tried enabling the anonymous bind and now also the directory browser
> (and all the login process) works as expected.
>
> Nicola
>
> Il 21/12/15 17:39, Cal Sawyer ha scritto:
>
> Thanks, John and Nicola
>
> Kerberos occurred to me as well late in the day yesterday.  Happily (?),
> knit works fine simply specifying the user in question with no need to
> suffix with the kerberos realm
>
> I did find that my test user had an expired password, which i fixed on the
> IPA server.  This was never flagged up under Linux, btw.  It has not change
> anything, however, other than not prompting for password changes that never
> take effect.  Funnily, it expired in the midst of testing - fun.
>
> I was mistaken when i said i was unable to log in - it turns out that it
> takes almost 10 minutes for a login from the frintend to complete - i just
> didn't wait long enough.  10 mins is of course unacceptable :)  "su - user"
> and "login user" fail outright after rejecting accept any user's password
>
> DNS is fine and i can resolve ldap and kerberos SRV records from the Mac
>
> In line with Nicola's experience, i can browse groups and users in the
> Directory Editor and all attributes appear spot on.
>
> Besides modding /etc/pam.d/authorization, adding a corrected
> edu.mit.kerberos to /LibraryPreferences and setting up the directory per
> linsec.ca, can anyone think of something i may have missed?  It's a real
> shame that the documentation on this stops around 5 years ago.
>
> IPA devs: is there anything i should be on the lookout for in the dirsrv
> or krb5 logs on the IPA master?  I've disabled the secondary to prevent
> replication from clouding the log events
>
> thanks, everyone
>
> Cal Sawyer | Systems Engineer | BlueBolt Ltd
> 15-16 Margaret Street | London W1W 8RW+44 (0)20 7637 5575 | www.blue-bolt.com
>
> On 21/12/15 07:57, Nicola Canepa wrote:
>
> Hello, I tried 2 weeks ago from Mavericks (OSX 10.9), but I had the
> opposite problem: kinit works fine, while I'm unable to see users with
> Directory Admin ((it always says it cant' connect, either with or without
> SSL)
> I disabled anonymous searches in 389-ds, by the way.
>
> Nicola
>
> Il 21/12/15 07:50, John Obaterspok ha scritto:
>
> Hi Cal,
>
> Does a kinit work from a terminal? Does it work if you use "kinit user" or
> just if you use "kinit user@REALM.suffix"
>
> -- john
>
>
> 2015-12-20 15:09 GMT+01:00 Cal Sawyer :
>
>> Hi, all
>>
>> I'm attempting to set up LDAP auth (against IPA server 4.10) from a OSX
>> 10.10.5 (Yosemite) client
>>
>> Using the excellent instructions at
>> 
>> http://linsec.ca/Using_FreeIPA_for_User_Authentication#Mac_OS_X_10.7.2F10.8%20%22Linsec.ca%20tutorial%20for%20connecting%20Mac%20OS%2010.7%20to%20IPA%20Server,
>> I've populated the specified files, d/l'd the cert, am able to configure
>> Users and Groups objects/attribs and browse both from within OSX's
>> Directory Utility.ldapsearch similarly returns the expected results.
>>
>> In spite of this, i'm unable to authenticate as any IPA-LDAP user on this
>> system
>>
>> dirsrv log on the ipa master shows no apparent errors - remote auth
>> attempts exit with "RESULT err=0 tag=101 nentries=1 etime=0", but tell the
>> truth, there so much stuff there and being rather inexperienced with LDAP
>> diags i might easily be missing something in the details
>>
>> The linsec.ca instructions were written in the 10.7-10.8 era so
>> something may have changed since.  Having said that, we've had no problems
>> authenticating against our existing OpenLDAP server (which IPA is slated to
>> replace) right up to 10.10.5 with no zero to our Directory Utility setup.
>>
>> Hoping someone here has some contemporary experience with OSX and IPA and
>> for whom this issue rings a bell?
>>
>> many thanks
>>
>> Cal Sawyer | Systems Engineer | BlueBolt Ltd
>> 15-16 Margaret Street | London W1W 8RW
>> +44 (0)20 7637 5575 <%2B44%20%280%2920%207637%205575> | www.blue-bolt.com
>>
>>
>> --
>> Manage your subscription for the Freeipa-users mailing list:
>> https://www.redhat.com/mailman/listinfo/freeipa-users
>> Go to http://freeipa.org for more info on the project
>>
>
>
>
>
> --
>
> Nicola Canepa
> T