On 04/20/2017 03:05 PM, Cox, Jason wrote:

-----Original Message-----
From: Rob Crittenden [mailto:rcrit...@redhat.com]
Sent: Wednesday, April 19, 2017 4:27 PM
To: Cox, Jason (U.S. Person) <jco...@harris.com>; freeipa-
us...@redhat.com
Subject: Re: [Freeipa-users] cannot add posix group or user

Cox, Jason wrote:
Hi all,



I had to reinstall my IPA setup, so I’m using 4.4 and am learning the
newer domain levels and topology features.

I’ve installed 3 servers.

I promoted one of the replicas to master and demoted the original
master to replica according to the documentation.
According to what documentation?

Note that they are all masters, some may just run different services and only
one has a few duties (like CRL generation).

Here: https://www.freeipa.org/page/Howto/Promote_CA_to_Renewal_and_CRL_Master
And here: 
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Linux_Domain_Identity_Authentication_and_Policy_Guide/server-roles.html#server-roles-promote-to-ca

Yes, I was referring to CRL master

And yes, I failed to continue reading 
https://access.redhat.com/documentation/en-US/Red_Hat_Enterprise_Linux/7/html/Linux_Domain_Identity_Authentication_and_Policy_Guide/
 to find what I needed to know concerning the id ranges. Sorry about that.


I ran into an issue with the original master no longer replicating, so
I performed an ipa-server-install –uninstall and removed the
host/server from IPA.
This is the where the problem started.

I re-setup the replica using ipa-client-install and then
ipa-replica-install, and had no errors reported in the output.

I then went into Web UI and setup replication agreements using the
topology graph page between the new replica and the previous replica
(the master/new replica agreements being setup by the replica install
script).



I then attempted to add a posix group account and got an operational
error message. This caused ldap to crash on the server I was
interfacing with.
If you are getting a core it would be very enlightening to get a stack trace
from that (you'll need to install the debuginfo package to get any really
useful data out of it).

I haven't had to get a core file from a systemd service before, so I did it the 
wrong way, but this is what I managed to get:

>From journalctl:
*** Error in `/usr/sbin/ns-slapd': free(): invalid pointer: 0x00007fbcd82f5fb0 
***
Apr 19 17:13:56 server1 ns-slapd[1892]: ======= Backtrace: =========
Apr 19 17:13:56 server1 ns-slapd[1892]: 
/lib64/libc.so.6(+0x7c503)[0x7fbd4522c503]
Apr 19 17:13:56 server1 ns-slapd[1892]: 
/lib64/libldap_r-2.4.so.2(ldap_mods_free+0x81)[0x7fbd46ba1a11]
Apr 19 17:13:56 server1 ns-slapd[1892]: 
/usr/lib64/dirsrv/libslapd.so.0(do_modify+0x7e0)[0x7fbd479f96a0]
Apr 19 17:13:56 server1 ns-slapd[1892]: 
/usr/sbin/ns-slapd(+0x1b9e0)[0x7fbd47ee29e0]
Apr 19 17:13:56 server1 ns-slapd[1892]: 
/lib64/libnspr4.so(+0x289bb)[0x7fbd45bd89bb]
Apr 19 17:13:56 server1 ns-slapd[1892]: 
/lib64/libpthread.so.0(+0x7dc5)[0x7fbd45578dc5]
Apr 19 17:13:56 server1 ns-slapd[1892]: 
/lib64/libc.so.6(clone+0x6d)[0x7fbd452a773d]


>From an eventual core and gdb (and not from the same crash as the journalctl 
output):
(gdb) bt
#0  __GI___libc_free (mem=0x41) at malloc.c:2929
#1  0x00007f87f6fca24c in ber_memvfree_x (vec=0x7f876c00a900, ctx=0x0) at 
memory.c:180
#2  0x00007f87f71f2a11 in ldap_mods_free (mods=0x7f876c001fb0, freemods=1) at 
free.c:94
#3  0x00007f87f804a6a0 in do_modify (pb=pb@entry=0x7f87b4ff0a90) at 
ldap/servers/slapd/modify.c:390
#4  0x00007f87f85339e0 in connection_dispatch_operation (pb=0x7f87b4ff0a90, 
op=0x7f87f931bf80, conn=0x7f87d82d0768) at ldap/servers/slapd/connection.c:627
#5  connection_threadmain () at ldap/servers/slapd/connection.c:1759
#6  0x00007f87f62299bb in _pt_root () from /lib64/libnspr4.so
#7  0x00007f87f5bc9dc5 in start_thread (arg=0x7f87b4ff1700) at 
pthread_create.c:308
#8  0x00007f87f58f873d in clone () at 
../sysdeps/unix/sysv/linux/x86_64/clone.S:113

Hi,

This is looking like the heap corruption and this backstack is unfortunately not enough to identify if it is a known/fixed one or not. This part of code (do_modify) was not recently changed regarding heap corruption and I would rather expect this thread to be the victim than responsible of it.
What 389-ds version are you running ?
We fixed recently a bug that could be the root cause (of course not 100% sure). Did you update 389-ds to the most recent one ?

Do you manage to reproduce this crash ?
For heap corruption, you may use valgrind but it could be too impacting for production performance.

regards
thierry

(gdb) bt full
#0  __GI___libc_free (mem=0x41) at malloc.c:2929
         ar_ptr = <optimized out>
         p = <optimized out>
         hook = 0x0
#1  0x00007f87f6fca24c in ber_memvfree_x (vec=0x7f876c00a900, ctx=0x0) at 
memory.c:180
         i = <optimized out>
#2  0x00007f87f71f2a11 in ldap_mods_free (mods=0x7f876c001fb0, freemods=1) at 
free.c:94
         i = <optimized out>
#3  0x00007f87f804a6a0 in do_modify (pb=pb@entry=0x7f87b4ff0a90) at 
ldap/servers/slapd/modify.c:390
         operation = 0x7f87f931bf80
         smods = {mods = 0x0, num_elements = 0, num_mods = 0, iterator = 0, 
free_mods = 0}
         ber = <optimized out>
         tag = <optimized out>
         len = 18446744073709551615
         normalized_mods = 0x7f876c001fb0
         mod = 0x0
         mods = 0x7f876c00c200
         last = 0x7f876c000e23 ""
         type = 0x0
         old_pw = 0x0
         rawdn = 0x7f876c000920 
"cn=svcaccount,cn=groups,cn=accounts,dc=MYDOMAIN"
         minssf_exclude_rootdse = <optimized out>
         ignored_some_mods = <optimized out>
         has_password_mod = <optimized out>
         pw_change = 0
         err = <optimized out>
#4  0x00007f87f85339e0 in connection_dispatch_operation (pb=0x7f87b4ff0a90, 
op=0x7f87f931bf80, conn=0x7f87d82d0768) at ldap/servers/slapd/connection.c:627
         minssf = 0
         minssf_exclude_rootdse = <optimized out>
---Type <return> to continue, or q <return> to quit---
         enable_nagle = 1
         pop_cork = 0
#5  connection_threadmain () at ldap/servers/slapd/connection.c:1759
         is_timedout = 0
         curtime = <optimized out>
         local_pb = {pb_backend = 0x7f87f8e1a070, pb_conn = 0x7f87d82d0768, 
pb_op = 0x7f87f931bf80, pb_plugin = 0x7f87f8c85c50, pb_opreturn = -1, pb_object 
= 0x0, pb_destroy_fn = 0x0,
           pb_requestor_isroot = 0, pb_config_fname = 0x0, pb_config_lineno = 
0, pb_config_argc = 0, pb_config_argv = 0x0, plugin_tracking = 0, 
pb_target_entry = 0x0,
           pb_existing_dn_entry = 0x7f876c00e880, pb_existing_uniqueid_entry = 
0x0, pb_parent_entry = 0x0, pb_newparent_entry = 0x0, pb_pre_op_entry = 0x0, 
pb_post_op_entry = 0x0,
           pb_seq_type = 0, pb_seq_attrname = 0x0, pb_seq_val = 0x0, 
pb_dbverify_dbdir = 0x0, pb_ldif_file = 0x0, pb_removedupvals = 0, 
pb_db2index_attrs = 0x0,
           pb_ldif2db_noattrindexes = 0, pb_ldif_printkey = 0, pb_instance_name 
= 0x0, pb_task = 0x0, pb_task_flags = 0, pb_mr_filter_match_fn = 0x0, 
pb_mr_filter_index_fn = 0x0,
           pb_mr_filter_reset_fn = 0x0, pb_mr_index_fn = 0x0, pb_mr_oid = 0x0, 
pb_mr_type = 0x0, pb_mr_value = 0x0, pb_mr_values = 0x0, pb_mr_keys = 0x0, 
pb_mr_filter_reusable = 0,
           pb_mr_query_operator = 0, pb_mr_usage = 0, 
pb_pwd_storage_scheme_user_passwd = 0x0, pb_pwd_storage_scheme_db_passwd = 0x0, 
pb_managedsait = 0, pb_internal_op_result = 0,
           pb_plugin_internal_search_op_entries = 0x0, 
pb_plugin_internal_search_op_referrals = 0x0, pb_plugin_identity = 0x0, 
pb_plugin_config_area = 0x0, pb_parent_txn = 0x0,
           pb_txn = 0x0, pb_txn_ruv_mods_fn = 0x7f87ea323470 
<replica_ruv_smods_for_op>, pb_dbsize = 0, pb_ldif_files = 0x0, pb_ldif_include 
= 0x0, pb_ldif_exclude = 0x0,
           pb_ldif_dump_replica = 0, pb_ldif_dump_uniqueid = 0, 
pb_ldif_generate_uniqueid = 0, pb_ldif_namespaceid = 0x0, pb_ldif_encrypt = 0, 
pb_operation_notes = 0, pb_slapd_argc = 0,
           pb_slapd_argv = 0x0, pb_slapd_configdir = 0x0, pb_ctrls_arg = 0x0, 
pb_dse_dont_add_write = 0, pb_dse_add_merge = 0, pb_dse_dont_check_dups = 0, 
pb_dse_is_primary_file = 0,
           pb_schema_flags = 0, pb_result_code = 0, pb_result_text = 0x0, 
pb_result_matched = 0x0, pb_nentries = 0, urls = 0x0, pb_import_entry = 0x0, 
pb_import_state = 0,
           pb_destroy_content = 0, pb_dse_reapply_mods = 0, 
pb_urp_naming_collision_dn = 0x0, pb_urp_tombstone_uniqueid = 0x0, 
pb_server_running = 0, pb_backend_count = 1,
           pb_pwpolicy_ctrl = 0, pb_vattr_context = 0x0, pb_substrlens = 0x0, 
pb_plugin_enabled = 0, pb_search_ctrls = 0x0, pb_mr_index_sv_fn = 0x0, 
pb_syntax_filter_normalized = 0,
           pb_syntax_filter_data = 0x0, pb_paged_results_index = 0, 
pb_paged_results_cookie = 0, pwdpolicy = 0x0, op_stack_elem = 0x7f87f8e24d30, 
pb_aci_target_check = 0,
           pb_pw_entry = 0x0}
         pb = 0x7f87b4ff0a90
         conn = 0x7f87d82d0768
         op = 0x7f87f931bf80
         tag = 102
         need_wakeup = 0
         thread_turbo_flag = <optimized out>
         ret = <optimized out>
         more_data = 0
---Type <return> to continue, or q <return> to quit---
         replication_connection = 0
         doshutdown = 0
         maxthreads = 5
         enable_nunc_stans = 0
         bypasspollcnt = <optimized out>
#6  0x00007f87f62299bb in _pt_root () from /lib64/libnspr4.so
No symbol table info available.
#7  0x00007f87f5bc9dc5 in start_thread (arg=0x7f87b4ff1700) at 
pthread_create.c:308
         __res = <optimized out>
         pd = 0x7f87b4ff1700
         now = <optimized out>
         unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140220833928960, 
-5233892399363934943, 0, 140220833929664, 140220833928960, 1, 
5211249063945286945, 5211388174174106913},
               mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = 
{prev = 0x0, cleanup = 0x0, canceltype = 0}}}
         not_first_call = <optimized out>
         pagesize_m1 = <optimized out>
         sp = <optimized out>
         freesize = <optimized out>
#8  0x00007f87f58f873d in clone () at 
../sysdeps/unix/sysv/linux/x86_64/clone.S:113
No locals.


I performed an ‘ipactl restart’ on the affected server and attempted
again with the same issue.

I tried adding a non-posix group and it was successful.



I found the dirsrv logs and see the error ‘dna-plugin - dna_pre_op: no
more values available!!’ which lead me to
https://www.redhat.com/archives/freeipa-users/2014-
February/msg00247.h
tml



Performing the ldapserch I see:

   dnaMaxValue is 1100

   dnaNextValue is 1101

   dnaThreshold is 500
Right. A master only gets a range when it needs one. In this case it needed
one after the master holding the entire range went away.

I also did ‘ipa idrange-find’, which shows:



---------------

1 range matched

---------------

   Range name: MYDOMAIN.COM_id_range

   First Posix ID of the range: 1946000000

   Number of IDs in the range: 200000

   Range type: local domain range

----------------------------

Number of entries returned 1

----------------------------





So now my question is what do I need to change to fix the issue?

I can do the ldapmodify to adjust the dnaMaxValue, but I don’t know
what I should be adjusting the idrange to?

I’d like to keep the idrange the same and just adjust the dnaMaxValue,
so would I need to change dnaMaxValue to 200000?
See https://blog-rcritten.rhcloud.com/?p=50

rob
Thank you.
Setting the id ranges manually fixed my problem.




--
Manage your subscription for the Freeipa-users mailing list:
https://www.redhat.com/mailman/listinfo/freeipa-users
Go to http://freeipa.org for more info on the project

Reply via email to