#33537: Cloning test database fails with mysql-client 8.x and older
mysql-server or
mariadb-server
-------------------------------------+-------------------------------------
Reporter: Stephen Finucane | Owner: nobody
Type: Bug | Status: new
Component: Database layer | Version: 4.1
(models, ORM) |
Severity: Normal | Resolution:
Keywords: | Triage Stage:
| Unreviewed
Has patch: 0 | Needs documentation: 0
Needs tests: 0 | Patch needs improvement: 0
Easy pickings: 0 | UI/UX: 0
-------------------------------------+-------------------------------------
Comment (by Stephen Finucane):
Reproducing from #34083.
I'm attempting to run tests in parallel using MariaDB. When running tests,
all access to the additional databases fail. Scrolling back to the top, I
see the following error messages:
{{{
❯ tox -e py38-django41 -- patchwork.tests.api.test_user
patchwork.tests.api.test_event
py38-django41 run-test-pre: PYTHONHASHSEED='631429049'
py38-django41 run-test: commands[0] | python
/home/patchwork/patchwork/manage.py test --noinput --parallel -v 2
--timing -- patchwork.tests.api.test_user patchwork.tests.api.test_event
Found 25 test(s).
Creating test database for alias 'default' ('test_patchwork')...
Operations to perform:
Synchronize unmigrated apps: django_filters, humanize, messages,
rest_framework, staticfiles
Apply all migrations: admin, auth, authtoken, contenttypes, patchwork,
sessions, sites
Synchronizing apps without migrations:
Creating tables...
Running deferred SQL...
Running migrations:
Applying contenttypes.0001_initial... OK
... {skipped} ...
Applying sites.0002_alter_domain_unique... OK
Cloning test database for alias 'default' ('test_patchwork')...
mysqldump: Couldn't execute 'SELECT COLUMN_NAME,
JSON_EXTRACT(HISTOGRAM, '$."number-of-buckets-specified"')
FROM information_schema.COLUMN_STATISTICS WHERE SCHEMA_NAME
= 'test_patchwork' AND TABLE_NAME = 'auth_group';': Unknown table
'COLUMN_STATISTICS' in information_schema (1109)
Cloning test database for alias 'default' ('test_patchwork')...
mysqldump: Couldn't execute 'SELECT COLUMN_NAME,
JSON_EXTRACT(HISTOGRAM, '$."number-of-buckets-specified"')
FROM information_schema.COLUMN_STATISTICS WHERE SCHEMA_NAME
= 'test_patchwork' AND TABLE_NAME = 'auth_group';': Unknown table
'COLUMN_STATISTICS' in information_schema (1109)
System check identified no issues (0 silenced).
...
}}}
The same issue happens on multiple Django versions. With a bit of hacking
on the code in the tox venv, I was able to inspect the command that Django
is actually executing:
{{{
Dumping! cmd = mysqldump --user=patchwork --host=localhost --routines
--events test_patchwork
}}}
Executing this locally, I see the same issue:
{{{
❯ MYSQL_PWD=password mysqldump --user=patchwork --host=localhost
--routines --events patchwork
-- MySQL dump 10.13 Distrib 8.0.30, for Linux (x86_64)
--
-- Host: localhost Database: patchwork
-- ------------------------------------------------------
-- Server version 5.5.5-10.5.16-MariaDB
/*!40101 SET @OLD_CHARACTER_SET_CLIENT=@@CHARACTER_SET_CLIENT */;
/*!40101 SET @OLD_CHARACTER_SET_RESULTS=@@CHARACTER_SET_RESULTS */;
/*!40101 SET @OLD_COLLATION_CONNECTION=@@COLLATION_CONNECTION */;
/*!50503 SET NAMES utf8mb4 */;
/*!40103 SET @OLD_TIME_ZONE=@@TIME_ZONE */;
/*!40103 SET TIME_ZONE='+00:00' */;
/*!40014 SET @OLD_UNIQUE_CHECKS=@@UNIQUE_CHECKS, UNIQUE_CHECKS=0 */;
/*!40014 SET @OLD_FOREIGN_KEY_CHECKS=@@FOREIGN_KEY_CHECKS,
FOREIGN_KEY_CHECKS=0 */;
/*!40101 SET @OLD_SQL_MODE=@@SQL_MODE, SQL_MODE='NO_AUTO_VALUE_ON_ZERO'
*/;
/*!40111 SET @OLD_SQL_NOTES=@@SQL_NOTES, SQL_NOTES=0 */;
--
-- Table structure for table `auth_group`
--
DROP TABLE IF EXISTS `auth_group`;
/*!40101 SET @saved_cs_client = @@character_set_client */;
/*!50503 SET character_set_client = utf8mb4 */;
CREATE TABLE `auth_group` (
`id` int(11) NOT NULL AUTO_INCREMENT,
`name` varchar(150) NOT NULL,
PRIMARY KEY (`id`),
UNIQUE KEY `name` (`name`)
) ENGINE=InnoDB DEFAULT CHARSET=latin1;
/*!40101 SET character_set_client = @saved_cs_client */;
--
-- Dumping data for table `auth_group`
--
LOCK TABLES `auth_group` WRITE;
/*!40000 ALTER TABLE `auth_group` DISABLE KEYS */;
/*!40000 ALTER TABLE `auth_group` ENABLE KEYS */;
UNLOCK TABLES;
mysqldump: Couldn't execute 'SELECT COLUMN_NAME,
JSON_EXTRACT(HISTOGRAM, '$."number-of-buckets-specified"')
FROM information_schema.COLUMN_STATISTICS WHERE SCHEMA_NAME
= 'patchwork' AND TABLE_NAME = 'auth_group';': Unknown table
'COLUMN_STATISTICS' in information_schema (1109)
❯ echo $?
2
}}}
I'm using Fedora 36 with the default packages:
{{{
❯ cat /etc/system-release
Fedora release 36 (Thirty Six)
❯ sudo dnf list installed | grep mariadb
mariadb-backup.x86_64 3:10.5.16-1.fc36
@updates
mariadb-common.x86_64 3:10.5.16-1.fc36
@updates
mariadb-connector-c.x86_64 3.2.7-1.fc36
@updates
mariadb-connector-c-config.noarch 3.2.7-1.fc36
@updates
mariadb-connector-c-devel.x86_64 3.2.7-1.fc36
@updates
mariadb-connector-c-doc.noarch 3.2.7-1.fc36
@updates
mariadb-cracklib-password-check.x86_64 3:10.5.16-1.fc36
@updates
mariadb-devel.x86_64 3:10.5.16-1.fc36
@updates
mariadb-errmsg.x86_64 3:10.5.16-1.fc36
@updates
mariadb-gssapi-server.x86_64 3:10.5.16-1.fc36
@updates
mariadb-server.x86_64 3:10.5.16-1.fc36
@updates
mariadb-server-utils.x86_64 3:10.5.16-1.fc36
@updates
❯ sudo dnf list installed | grep mysql
community-mysql.x86_64 8.0.30-2.fc36
@updates
community-mysql-common.x86_64 8.0.30-2.fc36
@updates
mysql-selinux.noarch 1.0.5-1.fc36
@updates
}}}
I had previously reported a similar issue using a combination of `mysql-
client` with an older version of `mysql-server` or `mariadb-server`. This
was reported as #33537. However, I'm now seeing the issue with the latest
versions of both the server and client. As noted in the issue, I can work
around this on the command line by adding the `--column-statistics=0`
argument to `mysqldump`, however, there doesn't appear to be any mechanism
to do this in Django. Alternatively, I can add the following to `my.cnf`
(or equivalent):
{{{
/etc/my.cnf.d/mysqldump.cnf
[mysqldump]
column-statistics=0
}}}
There's a chance that someone has packaged something wrong in Fedora land
and that I'm using an incompatible client and server combo (MySQL-derived
client, MariaDB server). However, it's surprising that the tests proceeded
to run despite the DB clone failing. As you can see above, `mysqldump`
returned a non-zero exit code so there doesn't appear to be any reason
Django couldn't have stopped immediately. I think it would make sense to
fail fast at a minimum. If we wanted, we could also add the `--column-
statistics=0` argument to our call to `mysqldump` (or provide a mechanism
for users to do so), however, this isn't compatible with the 5.7.x version
of `mysqldump` or the version provided by e.g. the `mariadb-client`
package on Ubuntu) so this would have to be a version-specific check.
--
Ticket URL: <https://code.djangoproject.com/ticket/33537#comment:4>
Django <https://code.djangoproject.com/>
The Web framework for perfectionists with deadlines.
--
You received this message because you are subscribed to the Google Groups
"Django updates" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to django-updates+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/django-updates/01070183c6e7c449-932d17ef-0645-49b2-89c8-ff6eaa3e890b-000000%40eu-central-1.amazonses.com.
