When I add the mysql-connector-java.jar to /usr/lib/jvm/java/jre/lib/ext without touching the CLASSPATH in /etc/sysconfig/cloudstack-management it's working fine as well.. So either something is broken in the tomcat class loading mechanism, either it's related to some environment specifics of mine..
Still digging deeper, keeping you guys posted.. Kind regards, Nick Livens On Thu, May 19, 2016 at 10:05 AM, Nick LIVENS <nick.liv...@nuagenetworks.net > wrote: > Now I'm totally clueless, perhaps I'm hitting a bug in tomcat? > Following the official documentation of Apache Tomcat 7.0 ( > http://tomcat.apache.org/tomcat-7.0-doc/jndi-datasource-examples-howto.html > ) > > Especially the following part : > "java.sql.DriverManager supports the service provider > <http://docs.oracle.com/javase/6/docs/api/index.html?java/sql/DriverManager.html> > mechanism. This feature is that all the available JDBC drivers that > announce themselves by providing a META-INF/services/java.sql.Driver file > are automatically discovered, loaded and registered, relieving you from the > need to load the database driver explicitly before you create a JDBC > connection. However, the implementation is fundamentally broken in all Java > versions for a servlet container environment. The problem is that > java.sql.DriverManager will scan for the drivers only once. > > The JRE Memory Leak Prevention Listener > <http://tomcat.apache.org/tomcat-7.0-doc/config/listeners.html> that is > included with Apache Tomcat solves this by triggering the drivers scan > during Tomcat startup. This is enabled by default. It means that only > libraries visible to the listener such as the ones in $CATALINA_BASE/lib will > be scanned for database drivers. If you are considering disabling this > feature, note that the scan would be triggered by the first web application > that is using JDBC, leading to failures when this web application is > reloaded and for other web applications that rely on this feature. > > Thus, the web applications that have database drivers in their WEB-INF/lib > directory > cannot rely on the service provider mechanism and should register the > drivers explicitly. > > The list of drivers in java.sql.DriverManager is also a known source of > memory leaks. Any Drivers registered by a web application must be > deregistered when the web application stops. Tomcat will attempt to > automatically discover and deregister any JDBC drivers loaded by the web > application class loader when the web application stops. However, it is > expected that applications do this for themselves via a > ServletContextListener." > I've checked all steps : > * My mysql-connector-java contains the META-INF/services/java.sql.Driver > file > * This JAR is located in /usr/share/cloudstack-management/lib/ > * CATALINA_BASE = /usr/share/cloudstack-management > > These are the 3 steps to make the JDBC drivers auto register themselves, > and this is not working. > I've also found a thread on stackoverflow where someone is hitting the > same issue - 2nd reply ( > http://stackoverflow.com/questions/20224222/how-to-register-oracle-jdbc-driver-on-tomcat-7-0-47-startup > ) > > Kind regards, > Nick Livens > > On Wed, May 18, 2016 at 6:25 PM, Simon Weller <swel...@ena.com> wrote: > >> Are you able to test this on a cleanly installed OS? >> >> It sounds like you've potentially got quite a few customizations and I >> think it would be helpful to see whether this can be reproduced on a clean >> install. >> >> ________________________________________ >> From: Nick LIVENS <nick.liv...@nuagenetworks.net> >> Sent: Wednesday, May 18, 2016 11:02 AM >> To: dev@cloudstack.apache.org >> Subject: Re: MySQL : No suitable driver found for jdbc:mysql >> >> I've digged a bit deeper, and I've got it working by explicitly loading >> the >> MySQL JDBC driver with the following code change in >> TransactionLegacy.java : >> >> static { >> try { >> Class.forName("com.mysql.jdbc.Driver"); >> } catch (ClassNotFoundException e) { >> throw new CloudRuntimeException("Failed to load MySQL JDBC >> driver", e); >> } >> >> // Initialize with assumed db.properties file >> initDataSource(DbProperties.getDbProperties()); >> } >> >> This is the old way of doing it (< JDBC 4.0) >> I know that from that version onwards that these drivers are loaded >> automatically, so I thought that our mysql-connector was outdated. >> I've replaced it with the latest, but without success. The issue is still >> there. >> >> I've got 2 solutions, but either way, it should work with the current >> master ACS without changing stuff manually. >> * This code change >> * Changing the /etc/sysconfig/cloudstack-management file again, so that >> CLASSPATH contains the mysql-connector-java.jar again >> >> Any help / suggestions / ideas are much appreciated. >> >> Kind regards, >> Nick Livens >> >> >> On Wed, May 18, 2016 at 4:43 PM, Nick LIVENS < >> nick.liv...@nuagenetworks.net> >> wrote: >> >> > Java : >> > [root@csc-1 ~]# java -version >> > java version "1.7.0_85" >> > OpenJDK Runtime Environment (rhel-2.6.1.2.el7_1-x86_64 u85-b01) >> > OpenJDK 64-Bit Server VM (build 24.85-b03, mixed mode) >> > >> > >> > Tomcat : >> > [root@csc-1 ~]# tomcat version >> > Server version: Apache Tomcat/7.0.54 >> > Server built: Mar 24 2015 07:49:05 >> > Server number: 7.0.54.0 >> > OS Name: Linux >> > OS Version: 3.10.0-229.7.2.el7.x86_64 >> > >> > >> > Environment : >> > [root@csc-1 ~]# cat /etc/*-release >> > NAME="Red Hat Enterprise Linux Server" >> > VERSION="7.1 (Maipo)" >> > ID="rhel" >> > ID_LIKE="fedora" >> > VERSION_ID="7.1" >> > PRETTY_NAME="Red Hat Enterprise Linux Server 7.1 (Maipo)" >> > ANSI_COLOR="0;31" >> > CPE_NAME="cpe:/o:redhat:enterprise_linux:7.1:GA:server" >> > HOME_URL="https://www.redhat.com/"; >> > BUG_REPORT_URL="https://bugzilla.redhat.com/"; >> > REDHAT_BUGZILLA_PRODUCT="Red Hat Enterprise Linux 7" >> > REDHAT_BUGZILLA_PRODUCT_VERSION=7.1 >> > REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux" >> > REDHAT_SUPPORT_PRODUCT_VERSION="7.1" >> > Red Hat Enterprise Linux Server release 7.1 (Maipo) >> > Red Hat Enterprise Linux Server release 7.1 (Maipo) >> > >> > [root@csc-1 ~]# cat /proc/version >> > Linux version 3.10.0-229.7.2.el7.x86_64 ( >> > mockbu...@x86-030.build.eng.bos.redhat.com) (gcc version 4.8.3 20140911 >> > (Red Hat 4.8.3-9) (GCC) ) >> > >> > >> > If you need any more information, feel free to ask. But as a stated >> > before, the mysql-connector-java.jar is loaded >> > [root@csc-1 ~]# lsof -p 14058 | grep mysql >> > java 14058 cloud mem REG 253,1 883899 134860304 >> > /usr/share/java/mysql-connector-java.jar >> > java 14058 cloud 38r REG 253,1 883899 134860304 >> > /usr/share/java/mysql-connector-java.jar >> > >> > Kind regards, >> > Nick Livens >> > >> > On Wed, May 18, 2016 at 4:17 PM, Simon Weller <swel...@ena.com> wrote: >> > >> >> It prevented /usr/share/cloudstack-mysql-ha/lib/*.jar from loaded. They >> >> need to be loaded together it seems and pulling the ha jar into the >> >> classpath doesn't work either. >> >> >> >> So this change functions for all the folks that have tested it thus far >> >> and passed CI, so I wonder why your environment is having problem with >> it. >> >> >> >> Can you give a bit of a run down on your build environment you're >> testing >> >> this on? >> >> >> >> ________________________________________ >> >> From: Nick LIVENS <nick.liv...@nuagenetworks.net> >> >> Sent: Wednesday, May 18, 2016 9:06 AM >> >> To: dev@cloudstack.apache.org >> >> Subject: Re: MySQL : No suitable driver found for jdbc:mysql >> >> >> >> Simon, >> >> >> >> If I look at the changes of the commit I've mentioned, I can only see a >> >> change for /usr/share/cloudstack-mysql-ha/lib/*.jar (*jar -> *.jar) >> >> The mysql-connector-java was already part of it. >> >> >> >> This is the entry in our catalina.properties : >> >> >> >> >> common.loader=${catalina.base}/lib,${catalina.base}/lib/*.jar,${catalina.home}/lib,${catalina.home}/lib/*.jar,/usr/share/java/mysql-connector-java.jar,/usr/share/cloudstack-mysql-ha/lib/*.jar >> >> >> >> So that's exactly the same. I've narrowed it down to this exact >> commit, so >> >> that's why I mentioned it. >> >> >> >> Any idea why the removal of the mysql-connector-java in the CLASSPATH >> >> might >> >> break it? >> >> >> >> Kind regards, >> >> Nick Livens >> >> >> >> On Wed, May 18, 2016 at 3:54 PM, Simon Weller <swel...@ena.com> wrote: >> >> >> >> > Nick, >> >> > >> >> > mysql-connector-java.jar was moved into the tomcat common.loader in >> >> > /etc/cloudstack/management/catalina.properties: >> >> > >> >> > >> >> > >> >> >> common.loader=${catalina.base}/lib,${catalina.base}/lib/*.jar,${catalina.home}/lib,${catalina.home}/lib/*.jar,/usr/share/java/mysql-connector-java.jar,/usr/share/cloudstack-mysql-ha/lib/*.jar >> >> > >> >> > This was done to fix a bug where the mysql-ha feature was broken. >> >> > >> >> > Does your common.loader look similar to the above? >> >> > >> >> > - Si >> >> > >> >> > >> >> > >> >> > >> >> > >> >> > ________________________________________ >> >> > From: Nick LIVENS <nick.liv...@nuagenetworks.net> >> >> > Sent: Wednesday, May 18, 2016 5:49 AM >> >> > To: dev@cloudstack.apache.org >> >> > Subject: Re: MySQL : No suitable driver found for jdbc:mysql >> >> > >> >> > Hey all, >> >> > >> >> > So I've figured it out what's causing the issue, in >> >> > commit c22659d76d73f00f41c13776c490e17a50aacd20, there has been a >> >> change in >> >> > the classpath in cloud-management.sysconfig >> >> > The change removes the mysql-connector-java.jar from the classpath. >> >> > >> >> > Before : >> >> > >> >> > >> >> >> CLASSPATH=/etc/cloudstack/management:/usr/share/cloudstack-common:/usr/share/cloudstack-management/setup:/usr/share/java/mysql-connector-java.jar >> >> > >> >> > After : >> >> > >> >> > >> >> >> CLASSPATH=/etc/cloudstack/management:/usr/share/cloudstack-common:/usr/share/cloudstack-management/setup >> >> > >> >> > Commit comment : Addresses CLOUDSTACK-9300 where the MySQL HA >> >> > StaticStrategy class fails to load successfully >> >> > >> >> > Does anyone have an idea why the mysql-connector-java.jar was removed >> >> from >> >> > the classpath? >> >> > >> >> > Thanks in advance! >> >> > >> >> > On Wed, May 18, 2016 at 8:40 AM, Paul Angus < >> paul.an...@shapeblue.com> >> >> > wrote: >> >> > >> >> > > Thanks Rohit, >> >> > > >> >> > > I see what you're saying. >> >> > > The conversation was actually based on your statement 'Nick, >> install >> >> > > libmysql-java or equivalent pkg' to which I asked why a user would >> >> have >> >> > to >> >> > > install an apparently additional component (Nick had already said >> that >> >> > > mysql-connector-java.jar was loaded). >> >> > > >> >> > > The conversation then spiralled out of control. >> >> > > >> >> > > Kind regards, >> >> > > >> >> > > Paul Angus >> >> > > >> >> > > paul.an...@shapeblue.com >> >> > > www.shapeblue.com >> >> > > 53 Chandos Place, Covent Garden, London WC2N 4HSUK >> >> > > @shapeblue >> >> > > >> >> > > >> >> > > >> >> > > -----Original Message----- >> >> > > From: Rohit Yadav [mailto:rohit.ya...@shapeblue.com] >> >> > > Sent: 18 May 2016 03:51 >> >> > > To: dev@cloudstack.apache.org >> >> > > Cc: dev@cloudstack.apache.org >> >> > > Subject: Re: MySQL : No suitable driver found for jdbc:mysql >> >> > > >> >> > > Paul, >> >> > > >> >> > > >> >> > > rohit.ya...@shapeblue.com >> >> > > www.shapeblue.com >> >> > > 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue >> >> > > >> >> > > >> >> > > On May 17 2016, at 11:33 pm, Paul Angus <paul.an...@shapeblue.com> >> >> > wrote: >> >> > > >> >> > > @Rohit, >> >> > > >> >> > > On CentOS 7 I get >> >> > > >> >> > > [root@localhost ~]# yum install libmysql-java >> >> > > >> >> > > This is incorrect, perhaps you did not read my email: >> >> > > "The mysql-connector-java (on centos) or libmysql-java (on debian) >> >> > package >> >> > > provides the jar dependency that understands jdbc and is necessary >> for >> >> > > CloudStack to communicate with MySQL, execute operations etc." >> >> > > >> >> > > The libmysql-java is the package name on debian/ubuntu, I said >> >> > > libmysql-java or 'equivalent' as I don't know what environment Nick >> >> was >> >> > > running. On CentOS, the same package that provides the mysql-java >> >> > > client/jar dependency is called 'mysql-connector-java'. >> >> > > >> >> > > The hullabulla raised around packaging dependency, is a non-issue. >> >> > > >> >> > > Based on what Nick has shared, his environment has oracle-java and >> not >> >> > > openjdk -- it is likely a case of missing classpath, JAVA_HOME and >> >> java >> >> > jdk >> >> > > directory path issue in the initd/systemd script. >> >> > > >> >> > > Regards. >> >> > > >> >> > > >> >> > > Loaded plugins: fastestmirror >> >> > > base | 3.6 kB 00:00:00 >> >> > > epel/x86_64/metalink | 25 kB 00:00:00 >> >> > > epel | 4.3 kB 00:00:00 >> >> > > extras | 3.4 kB 00:00:00 >> >> > > updates | 3.4 kB 00:00:00 >> >> > > (1/3): epel/x86_64/group_gz | 170 kB 00:00:00 >> >> > > (2/3): epel/x86_64/updateinfo | 555 kB 00:00:00 >> >> > > (3/3): epel/x86_64/primary_db | 4.1 MB 00:00:00 >> >> > > Loading mirror speeds from cached hostfile >> >> > > * base: mirrors.clouvider.net >> >> > > * epel: epel.mirrors.ovh.net >> >> > > * extras: centos.serverspace.co.uk >> >> > > * updates: mirror.as29550.net >> >> > > No package libmysql-java available. >> >> > > Error: Nothing to do >> >> > > [root@localhost ~]# >> >> > > >> >> > > Kind regards, >> >> > > >> >> > > Paul Angus >> >> > > >> >> > > paul.an...@shapeblue.com >> >> > > www.shapeblue.com >> >> > > 53 Chandos Place, Covent Garden, London WC2N 4HSUK >> >> > > @shapeblue >> >> > > >> >> > > -----Original Message----- >> >> > > From: Rohit Yadav [mailto:rohit.ya...@shapeblue.com] >> >> > > Sent: 17 May 2016 18:15 >> >> > > To: dev@cloudstack.apache.org >> >> > > Subject: Re: MySQL : No suitable driver found for jdbc:mysql >> >> > > >> >> > > Nick, assuming it's a CentOS environment can you run >> >> > > cloudstack-setup-databases (again), make sure that database IP and >> >> > cluster >> >> > > node IP parameters in db.properties are correct, and finally check >> >> that >> >> > the >> >> > > jar is in the classpath in the initd/systemd script (check >> JAVA_HOME, >> >> jdk >> >> > > dirs etc.). Also watchout for errors in the logs. >> >> > > >> >> > > Oracle-java is not available from default repositories, in that >> case >> >> you >> >> > > may need to fix JAVA_HOME and jdk paths in initd scripts. >> >> > > >> >> > > Regards. >> >> > > ________________________________________ >> >> > > From: Nick LIVENS <nick.liv...@nuagenetworks.net> >> >> > > Sent: 17 May 2016 18:34:38 >> >> > > To: dev@cloudstack.apache.org >> >> > > Subject: MySQL : No suitable driver found for jdbc:mysql >> >> > > >> >> > > Hi all, >> >> > > >> >> > > I'm facing the following issue when installing ACS 4.9.0 (current >> >> > master). >> >> > > java.sql.SQLException: No suitable driver found for >> >> > > >> >> > >> >> >> jdbc:mysql://localhost:3306/cloud?autoReconnect=true&prepStmtCacheSize=517&cachePrepStmts=true >> >> > > at >> >> java.sql.DriverManager.getConnection(DriverManager.java:596) >> >> > > at >> >> java.sql.DriverManager.getConnection(DriverManager.java:215) >> >> > > at >> >> > > >> >> > > >> >> > >> >> >> org.apache.commons.dbcp.DriverManagerConnectionFactory.createConnection(DriverManagerConnectionFactory.java:75) >> >> > > at >> >> > > >> >> > > >> >> > >> >> >> org.apache.commons.dbcp.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:582) >> >> > > at >> >> > > >> >> > > >> >> > >> >> >> org.apache.commons.pool.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:1188) >> >> > > at >> >> > > >> >> > > >> >> > >> >> >> org.apache.commons.dbcp.PoolingDataSource.getConnection(PoolingDataSource.java:106) >> >> > > at >> >> > > >> >> > > >> >> > >> >> >> com.cloud.utils.db.TransactionLegacy.getStandaloneConnectionWithException(TransactionLegacy.java:202) >> >> > > at >> >> com.cloud.utils.db.Merovingian2.<init>(Merovingian2.java:68) >> >> > > at >> >> > > >> com.cloud.utils.db.Merovingian2.createLockMaster(Merovingian2.java:88) >> >> > > >> >> > > First, I thought that the mysql-connector-java.jar was not loaded / >> >> > > installed correctly. But this jar is loaded : >> >> > > [root@csc-1 java]# lsof -p 14064 | grep jar | grep mysql >> >> > > java 14064 cloud mem REG 253,1 883899 134860304 >> >> > > /usr/share/java/mysql-connector-java.jar >> >> > > java 14064 cloud 38r REG 253,1 883899 134860304 >> >> > > /usr/share/java/mysql-connector-java.jar >> >> > > >> >> > > Tomcat specifics : >> >> > > [root@csc-1 java]# tomcat version >> >> > > Server version: Apache Tomcat/7.0.54 >> >> > > Server built: Mar 24 2015 07:49:05 >> >> > > Server number: 7.0.54.0 >> >> > > OS Name: Linux >> >> > > OS Version: 3.10.0-229.7.2.el7.x86_64 >> >> > > Architecture: amd64 >> >> > > JVM Version: 1.7.0_85-mockbuild_2015_07_11_12_24-b00 >> >> > > JVM Vendor: Oracle Corporation >> >> > > >> >> > > Does anyone have an idea why I might be facing this issue? >> >> > > >> >> > > Thanks! >> >> > > >> >> > > Kind regards, >> >> > > Nick Livens >> >> > > >> >> > > rohit.ya...@shapeblue.com >> >> > > www.shapeblue.com >> >> > > 53 Chandos Place, Covent Garden, London WC2N 4HSUK @shapeblue >> >> > > >> >> > >> >> >> > >> > >> > >
