Bug#996028: [debian-mysql] Bug#996028: #996028 InnoDB: corrupted TRX_NO after upgrading to 10.3.31
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On Fri, 2021-10-22 at 15:22 +0200, Ondrej Zary wrote: > With 10.3.29 running, I've dumped all databases (mysqldump --all-databases - > p >mysql.dump), then dropped all databases, stopped mariadb and deleted > /var/lib/mysql/ib*. > Then restarted mariadb, restored databases (source mysql.dump) and finally > upgraded to 10.3.31. Thanks, I did the same, and it seemed to work. Regards, - -- Yves-Alexis -BEGIN PGP SIGNATURE- iQEzBAEBCAAdFiEE8vi34Qgfo83x35gF3rYcyPpXRFsFAmFy2ToACgkQ3rYcyPpX RFsnKggA7XFwEwfLwE/paI4ThRlsP6GbXSUT193ivkC//qVQCMdHc6HJPFkuMFEa 7DhLkyE+QfNhzPQad2XRhEoQklpp98392q1QztWt+z5lgopuw8PgHQXeyXLB4in0 dfUcbCCylIWf/HV0qzt5lMOn8aIgZ0pQrEDjC4hYolkl2lyhq8N8zyu5DnXAlSXL L90/VZUmuM48YOMZ40W5OTs/LYNB7O+dmMAhM6J4gM3xyQ+//Nn33f89v86BFemD DtqGtsCr+7ZIOHeZmf/OAHLudZ8z0wvVGeO80OCxc95K8Vo503L+CmZI4h/QGA6J KRm6Pv1tpwHIe4tuE+327DHAo1Z3YQ== =+ZaT -END PGP SIGNATURE-
Bug#996028: [debian-mysql] Bug#996028: #996028 InnoDB: corrupted TRX_NO after upgrading to 10.3.31
On Friday 22 October 2021, Yves-Alexis Perez wrote: > On Thu, 2021-10-14 at 19:38 +0200, Ondrej Zary wrote: > > MDEV-15912: Remove traces of insert_undo > > > > Let us simply refuse an upgrade from earlier versions if the > > upgrade procedure was not followed. This simplifies the purge, > > commit, and rollback of transactions. > > > > Before upgrading to MariaDB 10.3 or later, a clean shutdown > > of the server (with innodb_fast_shutdown=1 or 0) is necessary, > > to ensure that any incomplete transactions are rolled back. > > The undo log format was changed in MDEV-12288. There is only > > one persistent undo log for each transaction. > > > Hi Ondrej, > it might be worth trying with that patch reverted, but I'm a bit confused at > the commit message. That seems to imply my server didn't have a clean shutdown > before upgrading to 10.3, which is possible but that upgrade is quite old by > now (afaict it's between Stretch and Buster), so I'm unsure what to do now. > > innodb_fast_shutdown=0 doesn't terminate on 10.3.25, and I'm not sure if there > are other ways to ensure a “clean shutdown”. > > Regards, With 10.3.29 running, I've dumped all databases (mysqldump --all-databases -p >mysql.dump), then dropped all databases, stopped mariadb and deleted /var/lib/mysql/ib*. Then restarted mariadb, restored databases (source mysql.dump) and finally upgraded to 10.3.31. So my problem is solved. I've also found the patch that's causing the problem. I'm not going to waste any more time on this. During many years of running mysql, upgrade just worked all the time. Looks like there are some quality problems with the developement process now. -- Ondrej Zary
Bug#996028: [debian-mysql] Bug#996028: #996028 InnoDB: corrupted TRX_NO after upgrading to 10.3.31
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On Thu, 2021-10-14 at 19:38 +0200, Ondrej Zary wrote: > MDEV-15912: Remove traces of insert_undo > > Let us simply refuse an upgrade from earlier versions if the > upgrade procedure was not followed. This simplifies the purge, > commit, and rollback of transactions. > > Before upgrading to MariaDB 10.3 or later, a clean shutdown > of the server (with innodb_fast_shutdown=1 or 0) is necessary, > to ensure that any incomplete transactions are rolled back. > The undo log format was changed in MDEV-12288. There is only > one persistent undo log for each transaction. Hi Ondrej, it might be worth trying with that patch reverted, but I'm a bit confused at the commit message. That seems to imply my server didn't have a clean shutdown before upgrading to 10.3, which is possible but that upgrade is quite old by now (afaict it's between Stretch and Buster), so I'm unsure what to do now. innodb_fast_shutdown=0 doesn't terminate on 10.3.25, and I'm not sure if there are other ways to ensure a “clean shutdown”. Regards, - -- Yves-Alexis -BEGIN PGP SIGNATURE- iQEzBAEBCAAdFiEE8vi34Qgfo83x35gF3rYcyPpXRFsFAmFynjAACgkQ3rYcyPpX RFtw9Qf/WcV91Ea5ltjqC4rzuNn877t0sjbdYx/4gESfevP+OY8lBboAckXQfejK FAMZIrRtQ6vmLiTDpYEZgBEqVuyPICFJ2kuK6ejxzYT3zpUqoXMTi21fUd2pivKF IVVHdMlFEgHfErGxN66Rv8m/OZUknfDAmkdAgKO0+WS+HuD/wuN7Y2zRCQBZ5PJg 8GJUZK+9O/7UdfHtxJvEja4U7B8G+bwQhKBNtgolOIrfapakB9b63KasJ18PVpQm TpkL8XdolAG/cvqxp6UYjVdatOTmceH/tME0xzbnad/hNxzR4MZKjUlCrZ8T0zgW WvXUZt1YsEivScpY15T32FELKPQeDQ== =92/m -END PGP SIGNATURE-
Bug#996028: [debian-mysql] Bug#996028: #996028 InnoDB: corrupted TRX_NO after upgrading to 10.3.31
e46f76c9749d7758765ba274a212cfc2dcf3eeb8 is the first bad commit commit e46f76c9749d7758765ba274a212cfc2dcf3eeb8 Author: Marko Mäkelä Date: Mon Jun 21 12:34:07 2021 +0300 MDEV-15912: Remove traces of insert_undo Let us simply refuse an upgrade from earlier versions if the upgrade procedure was not followed. This simplifies the purge, commit, and rollback of transactions. Before upgrading to MariaDB 10.3 or later, a clean shutdown of the server (with innodb_fast_shutdown=1 or 0) is necessary, to ensure that any incomplete transactions are rolled back. The undo log format was changed in MDEV-12288. There is only one persistent undo log for each transaction. :04 04 4bd1cf7f96bcfcfdeadbdcd6771e348e13b0d312 6b52363e618d955a49834ebcef6778430e6372cb M extra :04 04 25b37ceda117b9a05a200c6e3a8d4bca38e23bd5 dff451f0a7a6d6246334aace697b230b4174f2c1 M mysql-test :04 04 4a0352d498b9487cae46c6363d86603de0ccb361 3e6aa2377e89f28192e987f2f8655e3d866ab4be M storage -- Ondrej Zary
Bug#996028: [debian-mysql] Bug#996028: #996028 InnoDB: corrupted TRX_NO after upgrading to 10.3.31
Tested upstream mariadb packages. 10.3.30 works, 10.3.31 fails. -- Ondrej Zary
Bug#996028: [debian-mysql] Bug#996028: #996028 InnoDB: corrupted TRX_NO after upgrading to 10.3.31
On Thursday 14 October 2021, Yves-Alexis Perez wrote: > On Thu, 2021-10-14 at 00:03 -0700, Otto Kekäläinen wrote: > > Right. Here is for Buster: > > > > https://salsa.debian.org/mariadb-team/mariadb-10.3/-/commit/8ccf2240960cbb609cedfeb269df22d43ccbba21 > > https://salsa.debian.org/mariadb-team/mariadb-10.3/-/pipelines/302376 > > Unfortunately, with: > > ii libmariadb3:amd64 1:10.3.31-0+deb10u1+salsaci > amd64MariaDB database client library > ii mariadb-client 1:10.3.31-0+deb10u1+salsaci all > MariaDB database client (metapackage depending on the latest version) > ii mariadb-client-10.31:10.3.31-0+deb10u1+salsaci > amd64MariaDB database client binaries > ii mariadb-client-core-10.3 1:10.3.31-0+deb10u1+salsaci > amd64MariaDB database core client binaries > ii mariadb-common 1:10.3.31-0+deb10u1+salsaci all > MariaDB common metapackage > ii mariadb-server 1:10.3.31-0+deb10u1+salsaci all > MariaDB database server (metapackage depending on the latest version) > ii mariadb-server-10.31:10.3.31-0+deb10u1+salsaci > amd64MariaDB database server binaries > ii mariadb-server-core-10.3 1:10.3.31-0+deb10u1+salsaci > amd64MariaDB database core server files > > > I still get: > > 2021-10-14 9:38:48 0 [Note] InnoDB: Using Linux native AIO > 2021-10-14 9:38:48 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic > builtins > 2021-10-14 9:38:48 0 [Note] InnoDB: Uses event mutexes > 2021-10-14 9:38:48 0 [Note] InnoDB: Compressed tables use zlib 1.2.11 > 2021-10-14 9:38:48 0 [Note] InnoDB: Number of pools: 1 > 2021-10-14 9:38:48 0 [Note] InnoDB: Using SSE2 crc32 instructions > 2021-10-14 9:38:48 0 [Note] InnoDB: Initializing buffer pool, total size = > 128M, instances = 1, chunk size = 128M > 2021-10-14 9:38:48 0 [Note] InnoDB: Completed initialization of buffer pool > 2021-10-14 9:38:48 0 [Note] InnoDB: If the mysqld execution user is > authorized, page cleaner thread priority can be changed. See the man page of > setpriority(). > 2021-10-14 9:38:48 0 [ERROR] InnoDB: corrupted TRX_NO 10002001a6990af > 2021-10-14 9:38:48 0 [Note] InnoDB: Retry with innodb_force_recovery=5 > 2021-10-14 9:38:48 0 [ERROR] InnoDB: Plugin initialization aborted with > error Data structure corruption > 2021-10-14 9:38:49 0 [Note] InnoDB: Starting shutdown... > 2021-10-14 9:38:49 0 [ERROR] Plugin 'InnoDB' init function returned error. > 2021-10-14 9:38:49 0 [ERROR] Plugin 'InnoDB' registration as a STORAGE > ENGINE failed. > 2021-10-14 9:38:49 0 [Note] Plugin 'FEEDBACK' is disabled. > 2021-10-14 9:38:49 0 [ERROR] Unknown/unsupported storage engine: InnoDB > 2021-10-14 9:38:49 0 [ERROR] Aborting > > So maybe I do have a corrupted database or something but I'm unsure what to do > next (I don't think I have an older backups of the database). Also it still > works just fine with 10.3.25. I still get the error too. That patch was meant to fix a problem specific to BSD. -- Ondrej Zary
Bug#996028: [debian-mysql] Bug#996028: #996028 InnoDB: corrupted TRX_NO after upgrading to 10.3.31
> So maybe I do have a corrupted database or something but I'm unsure what to do > next (I don't think I have an older backups of the database). Also it still > works just fine with 10.3.25. Run it with the old database, make a logical dump (with mysqldump) and import that logical dump into a fresh new installation maybe?
Bug#996028: [debian-mysql] Bug#996028: #996028 InnoDB: corrupted TRX_NO after upgrading to 10.3.31
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On Thu, 2021-10-14 at 00:03 -0700, Otto Kekäläinen wrote: > Right. Here is for Buster: > > https://salsa.debian.org/mariadb-team/mariadb-10.3/-/commit/8ccf2240960cbb609cedfeb269df22d43ccbba21 > https://salsa.debian.org/mariadb-team/mariadb-10.3/-/pipelines/302376 Unfortunately, with: ii libmariadb3:amd64 1:10.3.31-0+deb10u1+salsaci amd64 MariaDB database client library ii mariadb-client 1:10.3.31-0+deb10u1+salsaci all MariaDB database client (metapackage depending on the latest version) ii mariadb-client-10.31:10.3.31-0+deb10u1+salsaci amd64 MariaDB database client binaries ii mariadb-client-core-10.3 1:10.3.31-0+deb10u1+salsaci amd64 MariaDB database core client binaries ii mariadb-common 1:10.3.31-0+deb10u1+salsaci all MariaDB common metapackage ii mariadb-server 1:10.3.31-0+deb10u1+salsaci all MariaDB database server (metapackage depending on the latest version) ii mariadb-server-10.31:10.3.31-0+deb10u1+salsaci amd64 MariaDB database server binaries ii mariadb-server-core-10.3 1:10.3.31-0+deb10u1+salsaci amd64 MariaDB database core server files I still get: 2021-10-14 9:38:48 0 [Note] InnoDB: Using Linux native AIO 2021-10-14 9:38:48 0 [Note] InnoDB: Mutexes and rw_locks use GCC atomic builtins 2021-10-14 9:38:48 0 [Note] InnoDB: Uses event mutexes 2021-10-14 9:38:48 0 [Note] InnoDB: Compressed tables use zlib 1.2.11 2021-10-14 9:38:48 0 [Note] InnoDB: Number of pools: 1 2021-10-14 9:38:48 0 [Note] InnoDB: Using SSE2 crc32 instructions 2021-10-14 9:38:48 0 [Note] InnoDB: Initializing buffer pool, total size = 128M, instances = 1, chunk size = 128M 2021-10-14 9:38:48 0 [Note] InnoDB: Completed initialization of buffer pool 2021-10-14 9:38:48 0 [Note] InnoDB: If the mysqld execution user is authorized, page cleaner thread priority can be changed. See the man page of setpriority(). 2021-10-14 9:38:48 0 [ERROR] InnoDB: corrupted TRX_NO 10002001a6990af 2021-10-14 9:38:48 0 [Note] InnoDB: Retry with innodb_force_recovery=5 2021-10-14 9:38:48 0 [ERROR] InnoDB: Plugin initialization aborted with error Data structure corruption 2021-10-14 9:38:49 0 [Note] InnoDB: Starting shutdown... 2021-10-14 9:38:49 0 [ERROR] Plugin 'InnoDB' init function returned error. 2021-10-14 9:38:49 0 [ERROR] Plugin 'InnoDB' registration as a STORAGE ENGINE failed. 2021-10-14 9:38:49 0 [Note] Plugin 'FEEDBACK' is disabled. 2021-10-14 9:38:49 0 [ERROR] Unknown/unsupported storage engine: InnoDB 2021-10-14 9:38:49 0 [ERROR] Aborting So maybe I do have a corrupted database or something but I'm unsure what to do next (I don't think I have an older backups of the database). Also it still works just fine with 10.3.25. Regards, - -- Yves-Alexis -BEGIN PGP SIGNATURE- iQEzBAEBCAAdFiEE8vi34Qgfo83x35gF3rYcyPpXRFsFAmFn33MACgkQ3rYcyPpX RFvQ/AgA0ijpQynfY/h6UsBhBKIpfDKUFB+Ti1xJY01RHU0ZxqjvxjYKeCH8CD8U +JEhFwTurzxS64tGGyS7jyT+9JUi4tcodtOWEk9I/hKKP4nC31IT7Ov9BQfDeJ7E wL82cA5/c92r3IZYs8AfKG8KWjVCrSCMMzjkVaPIuKvf9UYY9WlioPW7X55MihkE lIOXGk3jWp6LtT6QG8OOmoM5w/QmUKaOR5soPaFaQm/Km5KibvtY8nPEhhaIYe6x sSdJDqIo7uOt3DEXUujT8uRf3sd2O15G2DMv0A/6DX7Yb6zO03zQHfuCeZURm2p2 H7j+0p6DQWUOyKHuaMaAvtpM9hcbGQ== =BU6z -END PGP SIGNATURE-
Bug#996028: [debian-mysql] Bug#996028: #996028 InnoDB: corrupted TRX_NO after upgrading to 10.3.31
Right. Here is for Buster: https://salsa.debian.org/mariadb-team/mariadb-10.3/-/commit/8ccf2240960cbb609cedfeb269df22d43ccbba21 https://salsa.debian.org/mariadb-team/mariadb-10.3/-/pipelines/302376
Bug#996028: [debian-mysql] Bug#996028: #996028 InnoDB: corrupted TRX_NO after upgrading to 10.3.31
-BEGIN PGP SIGNED MESSAGE- Hash: SHA256 On Wed, 2021-10-13 at 19:06 -0700, Otto Kekäläinen wrote: > I added this patch to the packaging in > https://salsa.debian.org/mariadb-team/mariadb-10.5/-/commit/2a8f83531a2ff22a31c61b8bef28dabf77b25b78 > and once the CI completes (if it runs without errors) you will be able > to download the packages from the build artifacts published at > https://salsa.debian.org/mariadb-team/mariadb-10.5/-/pipelines/302326 > in step "build". Thanks! Unfortunately I'm rather looking for 10.3 packages since I'm still on Buster at the moment… Regards, - -- Yves-Alexis -BEGIN PGP SIGNATURE- iQEzBAEBCAAdFiEE8vi34Qgfo83x35gF3rYcyPpXRFsFAmFn0iMACgkQ3rYcyPpX RFvy+Af/eNK42v0kyygB4+LDf+ayBQ/sYNnFMRUiG3xDHil+Pi7r8tszrEgn1vl4 1jz7vHfMxDu/kzrOtetR9sVdJebocIigO8Wn6/NLWqq+Ut7bTQK1fOOYeXcowLeb Bc1lG6thlGVxKa+qTXSSivM/nGbl6JvunCg2HN0tauzy8nMnlNisUcssauvpvdun /nSPnsKUyLHwyU20FtruGowSxv2rVtAVTiA8cRF8aymm2kYe9NrmOSyJ+E6mrJQy onRmpQQqQNaGVr3bt7oakf2lNqjsb5j+cURblTNBhITriPv0vDIBeMXJHb3EokUk ZdTR3eeEV+qQxzyvHZ2TUadLpVGSHA== =69F+ -END PGP SIGNATURE-
Bug#996028: [debian-mysql] Bug#996028: #996028 InnoDB: corrupted TRX_NO after upgrading to 10.3.31
Hello! On Tue, Oct 12, 2021 at 4:51 AM Yves-Alexis Perez wrote: > > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA256 > > On Mon, 2021-10-11 at 11:27 +0200, Jan Korbel wrote: > > Maybe this: https://bugs.freebsd.org/bugzilla//show_bug.cgi?id=257728 > > If I read the bug correctly, it points to > https://jira.mariadb.org/projects/MDEV/issues/MDEV-26537 and the commit > https://github.com/MariaDB/server/commit/d09426f9e60fd93296464ec9eb5f9d85566437d3 > > I'm not sure I can rebuild the mariadb package myselve but I can surely test a > package if someone builds it. I added this patch to the packaging in https://salsa.debian.org/mariadb-team/mariadb-10.5/-/commit/2a8f83531a2ff22a31c61b8bef28dabf77b25b78 and once the CI completes (if it runs without errors) you will be able to download the packages from the build artifacts published at https://salsa.debian.org/mariadb-team/mariadb-10.5/-/pipelines/302326 in step "build".
