They are. And how do you start the controller? On Thu, Sep 24, 2020 at 2:19 PM Adam Goryachev < [email protected]> wrote:
> > On 24/9/20 15:30, Rene Peinthor wrote: > > Sounds weird, I don't think we changed anything with the database > initialization. > > Can you post your /etc/linstor/linstor.toml? > > Cheers, > Rene > > Hi, please see the contents below (I assume these are default settings): > > # Basic linstor configuration toml file > # For more options check documentation > > [db] > user = "linstor" > password = "linstor" > connection_url = "jdbc:h2:/var/lib/linstor/linstordb" > > Thanks, > Adam > > > On Wed, Sep 23, 2020 at 6:54 PM Adam Goryachev < > [email protected]> wrote: > >> Hi, >> >> After upgrading on one of my test nodes I got some errors, so I thought >> to upgrade the controller, and now I am unable to start the controller. >> >> Here is what I get in the system log: >> >> Sep 24 02:33:40 castle systemd[1]: Started LINSTOR Controller Service. >> Sep 24 02:33:41 castle Controller[1093]: LINSTOR, Module Controller >> Sep 24 02:33:41 castle Controller[1093]: Version: 1.9.0 >> (678acd24a8b9b73a735407cd79ca33a5e95eb2e2) >> Sep 24 02:33:41 castle Controller[1093]: Build time: >> 2020-09-23T10:27:49+00:00 >> Sep 24 02:33:41 castle Controller[1093]: Java Version: 11 >> Sep 24 02:33:41 castle Controller[1093]: Java VM: Debian, >> Version 11.0.8+10-post-Debian-1deb10u1 >> Sep 24 02:33:41 castle Controller[1093]: Operating system: Linux, >> Version 4.19.0-10-amd64 >> Sep 24 02:33:41 castle Controller[1093]: Environment: amd64, 2 >> processors, 1988 MiB memory reserved for allocations >> Sep 24 02:33:41 castle Controller[1093]: System components initialization >> in progress >> Sep 24 02:33:42 castle Controller[1093]: 02:33:42.355 [main] INFO >> LINSTOR/Controller - SYSTEM - ErrorReporter DB version 1 found. >> Sep 24 02:33:42 castle Controller[1093]: 02:33:42.359 [main] INFO >> LINSTOR/Controller - SYSTEM - Log directory set to: >> '/var/log/linstor-controller' >> Sep 24 02:33:42 castle Controller[1093]: 02:33:42.410 [main] WARN >> io.sentry.dsn.Dsn - *** Couldn't find a suitable DSN, Sentry operations >> will do nothing! See documentation: https://docs.sentry.io/clients/java/ >> *** >> Sep 24 02:33:42 castle Controller[1093]: 02:33:42.434 [Main] INFO >> LINSTOR/Controller - SYSTEM - Loading API classes started. >> Sep 24 02:33:43 castle Controller[1093]: 02:33:43.046 [Main] INFO >> LINSTOR/Controller - SYSTEM - API classes loading finished: 611ms >> Sep 24 02:33:43 castle Controller[1093]: 02:33:43.047 [Main] INFO >> LINSTOR/Controller - SYSTEM - Dependency injection started. >> Sep 24 02:33:43 castle Controller[1093]: 02:33:43.065 [Main] INFO >> LINSTOR/Controller - SYSTEM - Attempting dynamic load of extension module >> "com.linbit.linstor.spacetracking.ControllerSpaceTrackingModule" >> Sep 24 02:33:43 castle Controller[1093]: 02:33:43.065 [Main] INFO >> LINSTOR/Controller - SYSTEM - Extension module >> "com.linbit.linstor.spacetracking.ControllerSpaceTrackingModule" is not >> installed >> Sep 24 02:33:43 castle Controller[1093]: WARNING: An illegal reflective >> access operation has occurred >> Sep 24 02:33:43 castle Controller[1093]: WARNING: Illegal reflective >> access by com.google.inject.internal.cglib.core.$ReflectUtils$1 ( >> file:/usr/share/linstor-server/lib/guice-4.2.2.jar) to method >> java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain) >> Sep 24 02:33:43 castle Controller[1093]: WARNING: Please consider >> reporting this to the maintainers of >> com.google.inject.internal.cglib.core.$ReflectUtils$1 >> Sep 24 02:33:43 castle Controller[1093]: WARNING: Use >> --illegal-access=warn to enable warnings of further illegal reflective >> access operations >> Sep 24 02:33:43 castle Controller[1093]: WARNING: All illegal access >> operations will be denied in a future release >> Sep 24 02:33:44 castle Controller[1093]: 02:33:44.962 [Main] INFO >> LINSTOR/Controller - SYSTEM - Dependency injection finished: 1915ms >> Sep 24 02:33:45 castle Controller[1093]: 02:33:45.533 [Main] INFO >> LINSTOR/Controller - SYSTEM - Initializing authentication subsystem >> Sep 24 02:33:45 castle Controller[1093]: 02:33:45.736 [Main] INFO >> LINSTOR/Controller - SYSTEM - Starting service instance 'TimerEventService' >> of type TimerEventService >> Sep 24 02:33:45 castle Controller[1093]: 02:33:45.740 [Main] INFO >> LINSTOR/Controller - SYSTEM - Initializing the database connection pool >> Sep 24 02:33:46 castle Controller[1093]: 02:33:46.303 [Main] ERROR >> LINSTOR/Controller - SYSTEM - Database initialization error [Report number >> 5F6B78E5-00000-000000] >> Sep 24 02:33:46 castle Controller[1093]: 02:33:46.343 [Thread-2] INFO >> LINSTOR/Controller - SYSTEM - Shutdown in progress >> Sep 24 02:33:46 castle Controller[1093]: 02:33:46.344 [Thread-2] INFO >> LINSTOR/Controller - SYSTEM - Shutting down service instance >> 'TaskScheduleService' of type TaskScheduleService >> Sep 24 02:33:46 castle Controller[1093]: 02:33:46.345 [Thread-2] INFO >> LINSTOR/Controller - SYSTEM - Waiting for service instance >> 'TaskScheduleService' to complete shutdown >> Sep 24 02:33:46 castle Controller[1093]: 02:33:46.345 [Thread-2] INFO >> LINSTOR/Controller - SYSTEM - Shutting down service instance >> 'DatabaseService' of type DatabaseService >> Sep 24 02:33:46 castle Controller[1093]: 02:33:46.349 [Thread-2] INFO >> LINSTOR/Controller - SYSTEM - Waiting for service instance >> 'DatabaseService' to complete shutdown >> Sep 24 02:33:46 castle Controller[1093]: 02:33:46.350 [Thread-2] INFO >> LINSTOR/Controller - SYSTEM - Shutting down service instance >> 'TimerEventService' of type TimerEventService >> Sep 24 02:33:46 castle Controller[1093]: 02:33:46.351 [Thread-2] INFO >> LINSTOR/Controller - SYSTEM - Waiting for service instance >> 'TimerEventService' to complete shutdown >> Sep 24 02:33:46 castle Controller[1093]: 02:33:46.351 [Thread-2] INFO >> LINSTOR/Controller - SYSTEM - Shutdown complete >> Sep 24 02:33:46 castle systemd[1]: linstor-controller.service: Main >> process exited, code=exited, status=20/n/a >> Sep 24 02:33:46 castle systemd[1]: linstor-controller.service: Failed >> with result 'exit-code'. >> >> Here is /var/log/linstor-controller/ErrorReport-5F6B78E5-00000-000000.log >> >> ERROR REPORT 5F6B78E5-00000-000000 >> >> ============================================================ >> >> Application: LINBIT® LINSTOR >> Module: Controller >> Version: 1.9.0 >> Build ID: >> 678acd24a8b9b73a735407cd79ca33a5e95eb2e2 >> Build time: 2020-09-23T10:27:49+00:00 >> Error time: 2020-09-24 02:33:46 >> Node: castle >> >> ============================================================ >> >> Reported error: >> =============== >> >> Description: >> Database initialization error >> >> Category: LinStorException >> Class name: SystemServiceStartException >> Class canonical name: com.linbit.SystemServiceStartException >> Generated at: Method 'initialize', Source file >> 'DbConnectionPoolInitializer.java', Line #56 >> Error message: Database initialization error >> >> Call backtrace: >> >> Method Native Class:Line number >> initialize N >> com.linbit.linstor.dbcp.DbConnectionPoolInitializer:56 >> startSystemServices N >> com.linbit.linstor.core.ApplicationLifecycleManager:87 >> start N >> com.linbit.linstor.core.Controller:337 >> main N >> com.linbit.linstor.core.Controller:556 >> >> Caused by: >> ========== >> >> Category: RuntimeException >> Class name: LinStorDBRuntimeException >> Class canonical name: >> com.linbit.linstor.LinStorDBRuntimeException >> Generated at: Method 'setTransactionIsolation', >> Source file 'DbConnectionPool.java', Line #485 >> >> Error message: Failed to set transaction isolation >> >> Call backtrace: >> >> Method Native Class:Line number >> setTransactionIsolation N >> com.linbit.linstor.dbcp.DbConnectionPool:485 >> migrate N >> com.linbit.linstor.dbcp.DbConnectionPool:202 >> initialize N >> com.linbit.linstor.dbcp.DbConnectionPoolInitializer:50 >> startSystemServices N >> com.linbit.linstor.core.ApplicationLifecycleManager:87 >> start N >> com.linbit.linstor.core.Controller:337 >> main N >> com.linbit.linstor.core.Controller:556 >> >> Caused by: >> ========== >> >> Category: Exception >> Class name: JdbcSQLException >> Class canonical name: org.h2.jdbc.JdbcSQLException >> Generated at: Method 'getJdbcSQLException', Source >> file 'DbException.java', Line #357 >> >> Error message: Wrong user name or password >> [28000-197] >> >> Call backtrace: >> >> Method Native Class:Line number >> getJdbcSQLException N >> org.h2.message.DbException:357 >> get N >> org.h2.message.DbException:179 >> get N >> org.h2.message.DbException:155 >> get N >> org.h2.message.DbException:144 >> validateUserAndPassword N >> org.h2.engine.Engine:341 >> createSessionAndValidate N >> org.h2.engine.Engine:165 >> createSession N >> org.h2.engine.Engine:140 >> createSession N >> org.h2.engine.Engine:28 >> connectEmbeddedOrServer N >> org.h2.engine.SessionRemote:351 >> <init> N >> org.h2.jdbc.JdbcConnection:124 >> <init> N >> org.h2.jdbc.JdbcConnection:103 >> connect N org.h2.Driver:69 >> getConnection N >> java.sql.DriverManager:677 >> getConnection N >> java.sql.DriverManager:189 >> createConnection N >> org.apache.commons.dbcp2.DriverManagerConnectionFactory:123 >> makeObject N >> org.apache.commons.dbcp2.PoolableConnectionFactory:355 >> create N >> org.apache.commons.pool2.impl.GenericObjectPool:889 >> borrowObject N >> org.apache.commons.pool2.impl.GenericObjectPool:424 >> borrowObject N >> org.apache.commons.pool2.impl.GenericObjectPool:349 >> getConnection N >> org.apache.commons.dbcp2.PoolingDataSource:134 >> setTransactionIsolation N >> com.linbit.linstor.dbcp.DbConnectionPool:477 >> migrate N >> com.linbit.linstor.dbcp.DbConnectionPool:202 >> initialize N >> com.linbit.linstor.dbcp.DbConnectionPoolInitializer:50 >> startSystemServices N >> com.linbit.linstor.core.ApplicationLifecycleManager:87 >> start N >> com.linbit.linstor.core.Controller:337 >> main N >> com.linbit.linstor.core.Controller:556 >> >> Finally I found another log in /var/lib/linstor/linstordb.trace.db >> >> 2020-09-24 02:33:45 database: wrong user or password; user: "LINSTOR" >> org.h2.message.DbException: Wrong user name or password [28000-197] >> at org.h2.message.DbException.get(DbException.java:179) >> at org.h2.message.DbException.get(DbException.java:155) >> at org.h2.message.DbException.get(DbException.java:144) >> at org.h2.engine.Engine.openSession(Engine.java:107) >> at org.h2.engine.Engine.openSession(Engine.java:179) >> at org.h2.engine.Engine.createSessionAndValidate(Engine.java:157) >> at org.h2.engine.Engine.createSession(Engine.java:140) >> at org.h2.engine.Engine.createSession(Engine.java:28) >> at >> org.h2.engine.SessionRemote.connectEmbeddedOrServer(SessionRemote.java:351) >> at org.h2.jdbc.JdbcConnection.<init>(JdbcConnection.java:124) >> at org.h2.jdbc.JdbcConnection.<init>(JdbcConnection.java:103) >> at org.h2.Driver.connect(Driver.java:69) >> at >> java.sql/java.sql.DriverManager.getConnection(DriverManager.java:677) >> at >> java.sql/java.sql.DriverManager.getConnection(DriverManager.java:189) >> at >> org.apache.commons.dbcp2.DriverManagerConnectionFactory.createConnection(DriverManagerConnectionFactory.java:123) >> at >> org.apache.commons.dbcp2.PoolableConnectionFactory.makeObject(PoolableConnectionFactory.java:355) >> at >> org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:889) >> at >> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:424) >> at >> org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:349) >> at >> org.apache.commons.dbcp2.PoolingDataSource.getConnection(PoolingDataSource.java:134) >> at >> com.linbit.linstor.dbcp.DbConnectionPool.setTransactionIsolation(DbConnectionPool.java:477) >> at >> com.linbit.linstor.dbcp.DbConnectionPool.migrate(DbConnectionPool.java:202) >> at >> com.linbit.linstor.dbcp.DbConnectionPoolInitializer.initialize(DbConnectionPoolInitializer.java:50) >> at >> com.linbit.linstor.core.ApplicationLifecycleManager.startSystemServices(ApplicationLifecycleManager.java:87) >> at com.linbit.linstor.core.Controller.start(Controller.java:337) >> at com.linbit.linstor.core.Controller.main(Controller.java:556) >> Caused by: org.h2.jdbc.JdbcSQLException: Wrong user name or password >> [28000-197] >> at >> org.h2.message.DbException.getJdbcSQLException(DbException.java:357) >> ... 26 more >> >> >> Luckily, this is just a test environment, but I'd like to know if I >> messed up in my upgrade process somehow or this is a bug in the new >> version. I haven't downgraded or anything yet. >> >> Regards, >> Adam >> >> >> On 23/9/20 22:31, Rene Peinthor wrote: >> >> Hi! >> >> Here is a new minor Linstor release, this release contains more bug fixes >> than new features. >> Notable features are a new auto-unplace option, to reduce the >> replica-count of resources >> to the value set in the resource-group after a certain time. And >> error-reports can now be deleted >> from the REST-API (client). >> >> linstor-server 1.9.0 >> -------------------- >> * Added auto-"unplace", automatically removes resources if they exceed >> the replica count if `DrbdOptions/auto-unplace` is set >> * Added error-report deletion API >> * Expose DRBD promotion_score and may_promote via REST-API and server >> sent events >> * Prometheus metrics are now mostly cached and much faster to query >> * Send SD_NOTIFY READY to systemd if services are ready >> * OpenFlex resource are now correctly limited to 32 characters >> * Fix etcd modify net-interface (incorrect serialization) >> * Fix autoplacer also toggling diskless resources >> * Fix snapshot-shipping with more than 1 volume definitions >> * Fix zfs create options handling >> * Fixed recaching of lvm-filter >> * Fix missing client answer if drbd device doesn't become ready in >> reasonable time >> * REST-API v1.4.0 >> >> https://www.linbit.com/downloads/linstor/linstor-server-1.9.0.tar.gz >> >> Linstor PPA: >> https://launchpad.net/~linbit/+archive/ubuntu/linbit-drbd9-stack >> >> Cheers, >> Rene >> >> _______________________________________________ >> Star us on GITHUB: https://github.com/LINBIT >> drbd-user mailing >> [email protected]https://lists.linbit.com/mailman/listinfo/drbd-user >> >> _______________________________________________ >> Star us on GITHUB: https://github.com/LINBIT >> drbd-user mailing list >> [email protected] >> https://lists.linbit.com/mailman/listinfo/drbd-user >> >
_______________________________________________ Star us on GITHUB: https://github.com/LINBIT drbd-user mailing list [email protected] https://lists.linbit.com/mailman/listinfo/drbd-user
