Network throughput for mysql replication
Hi Guys I am once again having a weird issue... well weird to me anyway. We have a master,slave setup using mysql 5.7 and they are both connected on the same network segment through the same switch. During the weekend a HUGE amount of processing was done on the master, and thus resulted in the slave lagging and is currently on 213000 seconds behind master. While troubleshooting, I found the following : - MySQL io thread (relay log positions) are lagging far behind what is currently on master server. - SQL thread processes as the io thread are writing the relay log with no lag there. As a test , I disabled sql thread and let the io thread sync binlog data to relay log, while monitoring network traffic. The network traffic would max out at 6mb/s and as it needs to sync over 200Gb of binary logs, this seems to be taking too long. I then suspected a network issue, however when using SCP to copy data from the master server to slave server I get speeds of up to 80Mb/s for those copies. When I start the sql thread again after about 2 gb of data have been synced, then it catches up within a minute. So my question is this.. are there any options to mysql that would be causing it to limit the speed at which it sync binary logs. If not, how else can I speed up that process. FYI, the scp that was done was done to the same disks where the relay logs are on in order to also confirm that it is not a disk io issue, however that went more than good enough. So from what we can see the bottleneck is specifically related to slow sync over the network and seems to be only related to mysql. Any ideas here would be appreciated. Regards -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/mysql
Re: tcmalloc mysql 5.7.14
Hi Shawn Thank you for the response. I have in fact in the meantime installed jemmalloc libraries on a few servers and the issue with the memory is definitely gone when using jemmalloc library. I do still feel this is a bug though , as I am only able to produce this issue when using mysql 5.7 on ubuntu using the system memory libraries. Using other libraries like jemmalloc now seems to work 100% again, and in previous versions of mysql on same OS , using system libraries also works 100%. This issue I have is only present when using mysql 5.7. I have logged this as a bug, however simply got a reply that it is not a bug. but all the other tests as well as the new jemmalloc was all tested on the same machine and OS. Thank you for all the assistance though... Regards On Sat, May 13, 2017 9:41 pm, shawn l.green wrote: > (yes, I recognize how late this reply is) > > > On 5/8/2017 7:56 AM, Reindl Harald wrote: > >> >> >> Am 08.05.2017 um 13:51 schrieb Machiel Richards: >> >>> We are having an issue with memory allocations on mysql 5.7.14 >>> whereby mysql is not releasing the memory that is being allocated >>> during mysqldump processes. >>> >>> This has been logged as a bug with mysql dev team however they do >>> not see this as a bug and suggested we test using another malloc >>> library such as tcmalloc. >>> >>> However from what I can see, this is no longer included in mysql 5.7 >>> and thus I am trying to find out if anyone can tell me how to enable / >>> install this and how to tell mysql to use this library. >> >> does the oracle stuff not support "jemalloc" like MariaDB >> >> MariaDB even suppports "-DWITH_JEMALLOC=system" because it's not the >> job of random software ship and build random library sources in their >> tarballs which usually don't get much attention in case of updates >> (others than system packages) >> >> > > Yes, if your system has the jemalloc library on it, MySQL can use it. > One way to activate it is with this option to mysqld_safe. > > > https://dev.mysql.com/doc/refman/5.7/en/mysqld-safe.html#option_mysqld_sa > fe_malloc-lib > > And, you can control whether the InnoDB engine uses it's own memory > management routines or those provided by the operating system: > https://dev.mysql.com/doc/refman/5.7/en/innodb-performance-use_sys_malloc > .html > > > > > Just because we don't bundle a library with our software does not mean > that our end users cannot download one for their operating system from > other reputable sources (like the Google devs for tcmalloc, for example) > > -- > Shawn Green > MySQL Senior Principal Technical Support Engineer > Oracle USA, Inc. - Integrated Cloud Applications & Platform Services > Office: Blountville, TN > > > Become certified in MySQL! Visit https://www.mysql.com/certification/ > for details. > > -- > MySQL General Mailing List > For list archives: http://lists.mysql.com/mysql > To unsubscribe:http://lists.mysql.com/mysql > > > -- MySQL General Mailing List For list archives: http://lists.mysql.com/mysql To unsubscribe:http://lists.mysql.com/mysql
MySQL Connector/C++ 1.1.9 has been released
Dear MySQL Users, A new GA (general availability) version of MySQL Connector/C++ has been made available: MySQL Connector/C++ 1.1.9 GA. The MySQL Connector/C++ provides a C++ API for connecting client applications to the MySQL Server 5.5 or newer. You can download the production release at: http://dev.mysql.com/downloads/connector/cpp/1.1.html MySQL Connector C++ (Commercial) will be available for download on the My Oracle Support (MOS) website. This release will be available on eDelivery (OSDC) in next month's upload cycle. The MySQL driver for C++ offers an easy to use API derived from JDBC 4.0. MySQL Workbench has used it successfully for years. We have improved the driver since the last GA release. Please see the documentation and the CHANGES file in the source distribution for a detailed description of bugs that have been fixed. Bug descriptions are also listed below. Enjoy! Changes in MySQL Connector/C++ 1.1.9 (2017-05-16, General Availability) Compilation Notes * The Windows version of Connector/C++ Community is now built using the dynamic C++ runtime library (that is, with the /MD compiler option), with the following implications for users: + Target hosts running Windows applications that use Connector/C++ Community now need the Visual C++ Redistributable for Visual Studio 2013 (https://www.microsoft.com/en-us/download/details.aspx?id=40784) installed on them. + Client applications on Windows that use Connector/C++ Community should be compiled with the /MD compiler option. Security Notes * The linked OpenSSL library for Connector/C++ 1.1.9 Commercial has been updated to version 1.0.2k. For a description of issues fixed in this version, see http://www.openssl.org/news/vulnerabilities.html. This change does not affect the Oracle-produced MySQL Community build of Connector/C++, which uses the yaSSL library instead. Bugs Fixed * Values returned by getDouble() from DOUBLE table columns were truncated (decimal part missing) if the locale was set to fr_CA, which uses comma as the decimal separator. (Bug #17227390, Bug #69719) * Connections to localhost failed if the local server was bound only to its IPv6 interface. (Bug #17050354, Bug #69663) On Behalf of the MySQL/ORACLE RE Team Balasubramanian Kandasamy
