Bug#1006529: mariadb: FTBFS on hppa
For the record: After upload of MariaDB 1:10.11.2-3 the build passed. MTR was skipped because of DEB_BUILD_OPTIONS: nocheck https://buildd.debian.org/status/fetch.php?pkg=mariadb=hppa=1%3A10.11.2-3=168210=0 sbuild (Debian sbuild) 0.85.2 (11 March 2023) on mx3210 Kernel: Linux 6.1.24+ #1 SMP Sat Apr 15 15:43:47 UTC 2023 hppa (parisc64) Toolchain package versions: binutils_2.40-2 dpkg-dev_1.21.21 g++-12_12.2.0-14 gcc-12_12.2.0-14 libc6-dev_2.36-9 libstdc++-12-dev_12.2.0-14 libstdc++6_12.2.0-14 linux-libc-dev_6.1.20-2
Bug#1006529: mariadb fails on hppa
The MariaDB 1:10.11.2-1 build passed on a re-run at https://buildd.debian.org/status/package.php?p=mariadb, so all good now!
Bug#1006529:
2023-02-01 0:35:16 0 [Warning] mariadbd: Couldn't allocate 8388608 bytes (Large/HugeTLB memory page size 2097152); errno 22; continuing I believe it is a kernel bug. MariaDB is reading the page size from /sys/kernel/mm/hugepages. mapflag= MAP_PRIVATE | OS_MAP_ANON; mapflag|= my_bit_log2_size_t(large_page_size) << MAP_HUGE_SHIFT; ptr= mmap(NULL, aligned_size, PROT_READ | PROT_WRITE, mapflag, -1, 0); aligned_size is a memory size that is an increment of the page size. Per the mmap(2) man page, a ENOMEM, if memory cannot be allocated. EINVAL (22) is totally unexpected. From man page, the only condition satisfiable under the above conditions is length is too large (8M? seems unlikely), or not aligned on the page boundary. Given the page boundary was obtained from the /sys/kernel/mm/hugepages I'm out of explanations. Can you validate that the pages sizes on the directories in /sys/kernel/mm/hugepages are true to the architecture specification. I looked at kernel at mm/mmap.c and also can't see which path is returning EINVAL
Bug#1006529: mariadb fails on hppa
Hi Otto, Just FYI: mariadb fails on the hppa architecture, because there is a kernel bug (on parisc and probably other architectures) in the io_uring syscall. This is worked on upstream, e.g. this mail thread: https://lore.kernel.org/io-uring/507c7873--dbcb-c512-4659af486...@bell.net/T/#t We hope to get the kernel fixed in upcoming versions. Other buildds which run on qemu-user emulation will probably never support the io_uring syscall... Helge
Bug#1006529: marked as done (mariadb: FTBFS on hppa: builds, but test suite crashes immediately with "'aio write' returned OS error 0"), maybe io_uring related)
Control: reopen -1 Control: forwarded -1 https://jira.mariadb.org/browse/MDEV-30572 The upstream fix https://github.com/MariaDB/server/pull/2482 was backported via https://salsa.debian.org/mariadb-team/mariadb-server/-/commit/7ac10dee3b961cf69b330de23df5f8554450783e to latest Debian build. However, now the test suite fails to start at all in https://buildd.debian.org/status/fetch.php?pkg=mariadb=hppa=1%3A10.11.1-4=1676007600=0 ``` MariaDB Version 10.11.1-MariaDB-4 - SSL connections supported Using suites: main Collecting tests... Installing system database... - found 'core' (0/5) Core generated by '/<>/builddir/sql/mariadbd' Output from gdb follows. The first stack trace is from the failing thread. The following stack traces are from all threads (so the failing one is duplicated). -- warning: Can't open file anon_inode:[io_uring] which was expanded to anon_inode:[io_uring] during file-backed mapping note processing warning: Can't open file anon_inode:[io_uring] which was expanded to anon_inode:[io_uring] during file-backed mapping note processing [New LWP 31431] [New LWP 31427] [New LWP 31429] [New LWP 31428] [New LWP 31430] [Thread debugging using libthread_db enabled] Using host libthread_db library "/lib/hppa-linux-gnu/libthread_db.so.1". Core was generated by `/<>/builddir/sql/mariadbd --no-defaults --dis'. Program terminated with signal SIGABRT, Aborted. #0 0x43469c84 in my_register_filename (fd=1137958840, FileName=0x6 , type_of_file=3646115528, error_message_number=, MyFlags=) at ./mysys/my_open.c:140 140 ./mysys/my_open.c: No such file or directory. [Current thread is 1 (Thread 0xd9d34380 (LWP 31431))] #0 0x43469c84 in my_register_filename (fd=1137958840, FileName=0x6 , type_of_file=3646115528, error_message_number=, MyFlags=) at ./mysys/my_open.c:140 Backtrace stopped: Cannot access memory at address 0x7ab3 ``` The same upload also had other patches, so what we are seeing might be due to something else as well. Thus re-opening this bug and I see upstream Jira also remains open. Contributions from HPPA-experts welcome!
Bug#1006529: Info received (Bug#1006529: Info received (mariadb: FTBFS on hppa: builds, but test suite crashes immediately with "'aio write' returned OS error 0")))
For the record, latest build https://buildd.debian.org/status/fetch.php?pkg=mariadb=hppa=1%3A10.11.1-3=1675654714=0 passed but testsuite failed with: main.mysqldump 'innodb' w25 [ fail ] Test ended at 2023-02-06 03:25:25 CURRENT_TEST: main.mysqldump --- /<>/mysql-test/main/mysqldump.result 2022-11-14 18:10:21.0 + +++ /<>/mysql-test/main/mysqldump.reject 2023-02-06 03:25:24.020036990 + @@ -5524,7 +5524,7 @@ proc one DROP DATABASE bug25717383; -mariadb-dump: Got error: 2005: "Unknown server host 'unknownhost'" when trying to connect +mariadb-dump: Got error: 2002: "Can't connect to server on 'unknownhost'" when trying to connect mariadb-dump: Couldn't execute 'SHOW SLAVE STATUS': Server has gone away (2006) Usage: mariadb-dump [OPTIONS] database [tables] OR mariadb-dump [OPTIONS] --databases DB1 [DB2 DB3...]
Bug#1006529: Info received (mariadb: FTBFS on hppa: builds, but test suite crashes immediately with "'aio write' returned OS error 0"))
Latest build (https://buildd.debian.org/status/fetch.php?pkg=mariadb=hppa=1%3A10.11.1-2=1675231483=09 no longer fails on this but test suite runs, and fails on two tests: 1) main.func_json_notembedded w2 [ fail ] timeout after 7200 seconds Test ended at 2023-02-01 04:01:59 Test case timeout after 7200 seconds == /<>/builddir/mysql-test/var/2/log/func_json_notembedded.log == 1234567, 1234567, 1234567, 1234567, 1234567, 1234567, 1234567, 1234567, 1234567, 1234567, 1234567, 2345678 ] select json_loose(@arr); json_loose(@arr) [1234567, 1234567, 1234567, 1234567, 1234567, This is potentially related to https://jira.mariadb.org/browse/MDEV-30518 or https://jira.mariadb.org/browse/MDEV-27955. 2) main.large_pages 'innodb'w2 [ fail ] Test ended at 2023-02-01 00:35:17 CURRENT_TEST: main.large_pages Failed to start mysqld.1 mysqltest failed but provided no output - found 'core' (0/5) Trying 'dbx' to get a backtrace Trying 'lldb' to get a backtrace from coredump /<>/builddir/mysql-test/var/2/log/main.large_pages-innodb/mysqld.1/data/core Compressed file /<>/builddir/mysql-test/var/2/log/main.large_pages-innodb/mysqld.1/data/core - saving '/<>/builddir/mysql-test/var/2/log/main.large_pages-innodb/' to '/<>/builddir/mysql-test/var/log/main.large_pages-innodb/' Retrying test main.large_pages, attempt(2/3)... ***Warnings generated in error logs during shutdown after running tests: main.large_pages 2023-02-01 0:35:16 0 [Warning] mariadbd: Couldn't allocate 8388608 bytes (Large/HugeTLB memory page size 2097152); errno 22; continuing to smaller size 2023-02-01 0:35:16 0 [Warning] mariadbd: Couldn't allocate 6291456 bytes (Large/HugeTLB memory page size 2097152); errno 22; continuing to smaller size 2023-02-01 0:35:16 0 [Warning] mariadbd: Couldn't allocate 6291456 bytes (Large/HugeTLB memory page size 2097152); errno 22; continuing to smaller size 2023-02-01 0:35:16 0 [Warning] mariadbd: Couldn't allocate 4194304 bytes (Large/HugeTLB memory page size 2097152); errno 22; continuing to smaller size 2023-02-01 0:35:16 0 [Warning] mariadbd: Couldn't allocate 4194304 bytes (Large/HugeTLB memory page size 2097152); errno 22; continuing to smaller size 2023-02-01 0:35:16 0 [Warning] InnoDB: Retry attempts for reading partial data failed. 2023-02-01 0:35:16 0 [ERROR] InnoDB: Operating system error number 14 in a file operation. 2023-02-01 0:35:16 0 [ERROR] InnoDB: Error number 14 means 'Bad address' 2023-02-01 0:35:16 0 [ERROR] InnoDB: File (unknown): 'read' returned OS error 214. Cannot continue operation Attempting backtrace. You can use the following information to find out Filed the latter one as https://jira.mariadb.org/browse/MDEV-30572
Bug#1006529: mariadb: FTBFS on hppa: builds, but test suite crashes immediately with "'aio write' returned OS error 0")
Control: reassign -1 mariadb-10.6,mariadb Control: retitle -1 mariadb: FTBFS on hppa: builds, but test suite crashes immediately with "'aio write' returned OS error 0"), maybe io_uring related I reviewed recent builds of new MariaDB 10.11[1] and this same failure still exists. warning: Can't open file anon_inode:[io_uring] which was expanded to anon_inode:[io_uring] during file-backed mapping note processing ... [ERROR] InnoDB: File ./ibdata1: 'aio write' returned OS error 0. Cannot continue operation For suggestions to fix, Merge Requests[2] on Salsa are very welcome! [1] https://buildd.debian.org/status/package.php?p=mariadb [2] https://salsa.debian.org/mariadb-team/mariadb-server/-/wikis/Contributing-to-MariaDB-packaging-in-Debian
Bug#1006529: mariadb-10.6: FTBFS on hppa: builds, but test suite crashes immediately with "'aio write' returned OS error 0"
Source: mariadb-10.6 Version: 1:10.6.7-1 Tags: upstream, confirmed, help, ftbfs User: debian-h...@lists.debian.org Usertags: hppa Builds on hppa succeed, but as the mariadbd binary crashes immediately on start: mysql-test-run: *** ERROR: Error executing mariadbd --bootstrap Could not install system database from /<>/builddir/mysql-test/var/log/bootstrap.sql The /<>/builddir/mysql-test/var/log/bootstrap.log file contains: /<>/builddir/sql/mariadbd --no-defaults --disable-getopt-prefix-matching --bootstrap --basedir=/<> --datadir=/<>/builddir/mysql-test/var/install.db --plugin-dir=/<>/builddir/mysql-test/var/plugins --default-storage-engine=myisam --loose-skip-plugin-feedback --loose-skip-plugin-innodb --loose-skip-plugin-innodb-buffer-page-lru --loose-skip-plugin-innodb-buffer-pool-stats --loose-skip-plugin-innodb-cmp --loose-skip-plugin-innodb-cmp-per-index --loose-skip-plugin-innodb-cmp-per-index-reset --loose-skip-plugin-innodb-cmp-reset --loose-skip-plugin-innodb-cmpmem --loose-skip-plugin-innodb-cmpmem-reset --loose-skip-plugin-innodb-ft-being-deleted --loose-skip-plugin-innodb-ft-config --loose-skip-plugin-innodb-ft-default-stopword --loose-skip-plugin-innodb-ft-deleted --loose-skip-plugin-innodb-ft-index-cache --loose-skip-plugin-innodb-ft-index-table --loose-skip-plugin-innodb-sys-columns --loose-skip-plugin-innodb-sys-fields --loose-skip-plugin-innodb-sys-foreign --loose-skip-plugin-innodb-sys-foreign-cols --loose-skip-plugin-innodb-sys-indexes --loose-skip-plugin-innodb-sys-tables --loose-skip-plugin-innodb-sys-tablespaces --loose-skip-plugin-innodb-sys-tablestats --loose-skip-plugin-innodb-sys-virtual --loose-skip-plugin-innodb-tablespaces-encryption --loose-skip-plugin-partition --loose-skip-plugin-sequence --loose-skip-plugin-thread-pool-groups --loose-skip-plugin-thread-pool-queues --loose-skip-plugin-thread-pool-stats --loose-skip-plugin-thread-pool-waits --loose-skip-plugin-unix-socket --loose-skip-plugin-user-variables --loose-innodb --loose-innodb-log-file-size=10M --disable-sync-frm --tmpdir=/<>/builddir/mysql-test/var/tmp/ --core-file --console --character-set-server=latin1 --lc-messages-dir=/<>/builddir/sql/share/ --character-sets-dir=/<>/sql/share/charsets 2022-02-25 13:18:37 0 [Note] /<>/builddir/sql/mariadbd (server 10.6.7-MariaDB-2~exp1) starting as process 3949 ... 2022-02-25 13:18:37 0 [Warning] Could not increase number of max_open_files to more than 1024 (request: 32186) 2022-02-25 13:18:37 0 [Warning] Changed limits: max_open_files: 1024 max_connections: 151 (was 151) table_cache: 421 (was 2000) 2022-02-25 13:18:37 0 [Note] Plugin 'partition' is disabled. 2022-02-25 13:18:37 0 [Note] Plugin 'SEQUENCE' is disabled. 2022-02-25 13:18:37 0 [Note] InnoDB: The first data file './ibdata1' did not exist. A new tablespace will be created! 2022-02-25 13:18:37 0 [Note] InnoDB: Compressed tables use zlib 1.2.11 2022-02-25 13:18:37 0 [Note] InnoDB: Number of pools: 1 2022-02-25 13:18:37 0 [Note] InnoDB: Using generic crc32 instructions 2022-02-25 13:18:37 0 [Note] InnoDB: Using liburing 2022-02-25 13:18:37 0 [Note] InnoDB: Initializing buffer pool, total size = 134217728, chunk size = 134217728 2022-02-25 13:18:37 0 [Note] InnoDB: Completed initialization of buffer pool 2022-02-25 13:18:37 0 [Note] InnoDB: Setting O_DIRECT on file ./ibdata1 failed 2022-02-25 13:18:37 0 [Note] InnoDB: Setting file './ibdata1' size to 12 MB. Physically writing the file full; Please wait ... 2022-02-25 13:18:37 0 [Note] InnoDB: File './ibdata1' size is now 12 MB. 2022-02-25 13:18:37 0 [Note] InnoDB: Setting log file ./ib_logfile101 size to 10485760 bytes 2022-02-25 13:18:38 0 [ERROR] InnoDB: File ./ibdata1: 'aio write' returned OS error 0. Cannot continue operation 220225 13:18:38 [ERROR] mysqld got signal 6 ; This could be because you hit a bug. It is also possible that this binary or one of the libraries it was linked against is corrupt, improperly built, or misconfigured. This error can also be caused by malfunctioning hardware. To report this bug, see https://mariadb.com/kb/en/reporting-bugs We will try our best to scrape up some info that will hopefully help diagnose the problem, but since we have already crashed, something is definitely wrong and this may fail. Server version: 10.6.7-MariaDB-2~exp1 key_buffer_size=134217728 read_buffer_size=131072 max_used_connections=0 max_threads=153 thread_count=0 It is possible that mysqld could use up to key_buffer_size + (read_buffer_size + sort_buffer_size)*max_threads = 466655 K bytes of memory Hope that's ok; if not, decrease some variables in the equation. Thread pointer: 0x0 Attempting backtrace. You can use the following information to find out where mysqld died. If you see no messages after this, something went terribly wrong... stack_bottom = 0x0 thread_stack 0x49000 /<>/builddir/sql/mariadbd(my_print_stacktrace-0xb5a76b42)[0x4203d89c] The manual page at https://mariadb.com/kb/en/how-to-produce-a-full-stack-trace-for-mysqld/ contains information
