Hello all, I am testing handlersockets performance in a 5.5.28-29.1-log (Percona) server. These are the enabled options:
loose_handlersocket_port = 9998 loose_handlersocket_port_wr = 9999 loose_handlersocket_threads = 48 loose_handlersocket_threads_wr = 1 innodb_spin_wait_delay=0 The machine has 24 (Xeon - 2.00GHz) cores and 64GB RAM. We are using bonding to make sure the ethernets aren't limiting us here (we get around 90Mbps) We are able to handler around 500K requests per second using handler socket plugin. Even though it looks pretty impressive number, it's still not close to the 750K ones Yoshinori is able to get ( http://yoshinorimatsunobu.blogspot.com.es/2010/10/using-mysql-as-nosql-story-for.html ) The machine is acting a normal slave in a cluster - receiving normal traffic from our site (we do this on purpose to see how many requests we can handle in a normal workload environment) Obviously we're not expecting to get similar numbers as our tests aren't the same. However, doing a bit of profiling to try to determine what's the bottleneck here we've seen this: CPU: Intel Architectural Perfmon, speed 2000.26 MHz (estimated) Counted CPU_CLK_UNHALTED events (Clock cycles when not halted) with a unit mask of 0x00 (No unit mask) count 100000 samples % image name symbol name *2163868 24.2065 mysqld mutex_delay* 694540 7.7696 mysqld build_template(row_prebuilt_struct*, THD*, TABLE*, unsigned int) 626405 7.0074 mysqld buf_page_get_gen 607664 6.7978 mysqld rec_get_offsets_func 480937 5.3801 mysqld cmp_dtuple_rec_with_match 412081 4.6098 mysqld btr_cur_search_to_nth_level 365402 4.0876 mysqld rec_init_offsets 356064 3.9832 mysqld page_cur_search_with_match 310819 3.4770 mysqld row_search_for_mysql 274248 3.0679 mysqld row_sel_store_mysql_rec 208466 2.3320 mysqld my_pthread_fastmutex_lock 185853 2.0791 mysqld ha_innobase::index_read(unsigned char*, unsigned char const*, unsigned int, ha_rkey_function) 182939 2.0465 mysqld pfs_mutex_enter_func 154369 1.7269 mysqld mtr_memo_slot_release 138558 1.5500 mysqld page_check_dir 131622 1.4724 mysqld dict_index_copy_types 101162 1.1317 mysqld srv_conc_force_exit_innodb 72754 0.8139 mysqld ha_innobase::change_active_index(unsigned int) 65191 0.7293 mysqld my_long10_to_str_8bit 62574 0.7000 mysqld btr_pcur_store_position 61900 0.6925 mysqld pfs_mutex_exit_func 51889 0.5805 mysqld Field_long::pack_length() const 49073 0.5490 mysqld srv_conc_enter_innodb 44079 0.4931 mysqld ha_innobase::init_table_handle_for_HANDLER() 38998 0.4363 mysqld Field_tiny::pack_length() const 38868 0.4348 mysqld rec_copy_prefix_to_buf 36292 0.4060 mysqld ha_innobase::innobase_get_index(unsigned int) That mutex_delay is eating quite a big % of the time. I have not been able to find what is that related to. Does anyone has a clue about what's it and if there's a way to improve and overcome it? Cheers Manuel.
