Hi,
We are currently engaged into a performance analysis where we are analyzing
performance for User Add, Update, Authentication operations. The testing
has been carried out in a following environment with 500 concurrency and
users up to 10 million.
*Environment :*
m3.2xlarge ( 8 core, 30GB, SSD 2x80 GB) 3 instances.
MySQL 5.7
Ubuntu 14.04
Openldap-2.4.31
IS 5.1.0
In order to optimize the MYSQL server, following server parameters have
been tuned accordingly. We have referred MYSQL documentation [1] as well as
have performed analysis using several MYSQL tuners in [2].
(1) *max_connections : 1000* (The maximum permitted number of simultaneous
client connections.)
(2) *join_buffer_size : 259968* (The minimum size of the buffer that is
used for plain index scans, range index scans, and joins that do not use
indexes and thus perform full table scans.)
(3) *innodb_buffer_pool_size : 5207959552* (size of the memory area where
InnoDB caches table and index data)
(4) *innodb_log_buffer_size : 16777216* (size of the buffer for
transactions that have not been committed yet)
(5) *innodb_buffer_pool_instances : 1* (The number of buffer pool
instances. According to the mysql documentation[1], on systems with a large
amount of memory, we can improve concurrency by dividing the buffer pool
into multiple buffer pool instances. But couldn't change since it's a read
only variable)
(6) *key_buffer_size : 384000000* (size of the buffer used for index
blocks)
(7) *table_open_cache : 4000* (The number of open tables for all threads)
(8) *sort_buffer_size : 4000000* (Each session that must perform a sort
allocates a buffer of this size)
(9) *read_buffer_size : 1000000* (Each thread that does a sequential scan
for a table allocates a buffer of this size for each table it scans. If we
do many sequential scans, we might want to increase this value)
(10) *query_cache_type : 0 *
(11) *query_cache_limit : 1048576* (Do not cache results that are larger
than this number of bytes)
(12) *query_cache_size : 1048576* (The amount of memory allocated for
caching query results)
(13) *thread_stack : 262144* (The stack size for each thread)
(14) *net_buffer_length : 16384* (Each client thread is associated with a
connection buffer and result buffer. Both begin with a size given by
net_buffer_length but are dynamically enlarged up to max_allowed_packet
bytes as needed)
(15) *max_allowed_packet : 4194304* (The maximum size of one packet or any
generated/intermediate string)
(16) *thread_cache_size : 30* (no of threads the server should cache for
reuse)
IS has been configured as follows to optimize the performance.
(1) JVM Heap Settings (-Xms -Xmx) changed as follows:
*Xms : 2g *
*Xmx : 2g *
(2) Removed following entry from
<IS_HOME>/repository/conf/tomcat/catalina-server.xml to disable http access
logs.
<Valve className="org.apache.catalina.valves.AccessLogValve"
directory="${carbon.home}/repository/logs" prefix="http_access_"
suffix=".log" pattern="combined" />
(3) Tuned following parameters in axis2client.xml file.
<parameter name="*defaultMaxConnPerHost*">1000</parameter>
<parameter name="*maxTotalConnections*">30000</parameter>
(4) Added following additional parameters to optimize database connection
pool.
<Property name="*maxWait*">60000</Property>
<Property name="*maxActive*">600</Property>
<Property name="*initialSize*">20</Property>
(5) Tuning Tomcat parameters in
<IS_HOME>/repository/conf/tomcat/catalina-server.xml.
*acceptorThreadCount = 8 *
*maxThreads="750" *
*minSpareThreads="150" *
*maxKeepAliveRequests="600" *
*acceptCount="600"*
JMeter has been configured as follows to optimize the performance.
(1) JVM Heap Settings (-Xms -Xmx) changed as follows:
*Xms : 1g *
*Xmx : 1g *
We were able to optimize the environment up to some level. But* currently
the TPS is dropping from the initial TPS 1139.5/s to 198.1/s in around
6100000 user count.(User Add)*
Appreciate your help on figuring out whether we need to do any
modifications to the optimizations in MYSQL, IS and JMeter servers or to
identify the exact issue for this sudden TPS dropping.
[1] http://dev.mysql.com/doc/refman/5.7/en/optimizing-server.html
[2] http://www.askapache.com/mysql/mysql-performance-tuning.html
Thanks and Regards
--
Indunil Upeksha Rathnayake
Software Engineer | WSO2 Inc
Email indu...@wso2.com
Mobile 0772182255
_______________________________________________
Dev mailing list
Dev@wso2.org
http://wso2.org/cgi-bin/mailman/listinfo/dev
