Bug#1032958: rsyslog: Enable Large File Support (LFS)
On 3/14/23 19:20, Michael Biebl wrote: Am 14.03.23 um 19:13 schrieb Helge Deller: On 3/14/23 19:06, Helge Deller wrote: That flag is then used only for one C-file in the subdir contrib/omhiredis. Which flag? I meant the compiler preprocessor defines: The c-files should be compiled with: -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE Example: In plugins/impstats/impstats.c the glibc function readdir() is used. Without the "_FILE_OFFSET_BITS=64" define this readdir() will suddenly/randomly fail at runtime if the directory (or the files inside the directory) are stored at an inode with doesn't fit into 32-bits. The generated config.h contains /* Number of bits in a file offset, on hosts where this is settable. */ #define _FILE_OFFSET_BITS 64 and plugins/impstats/impstats.c includes config.h Seems you are correct. My request might be bogus. Will check further. Helge
Bug#1032958: rsyslog: Enable Large File Support (LFS)
Am 14.03.23 um 19:13 schrieb Helge Deller: On 3/14/23 19:06, Helge Deller wrote: That flag is then used only for one C-file in the subdir contrib/omhiredis. Which flag? I meant the compiler preprocessor defines: The c-files should be compiled with: -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE Example: In plugins/impstats/impstats.c the glibc function readdir() is used. Without the "_FILE_OFFSET_BITS=64" define this readdir() will suddenly/randomly fail at runtime if the directory (or the files inside the directory) are stored at an inode with doesn't fit into 32-bits. The generated config.h contains /* Number of bits in a file offset, on hosts where this is settable. */ #define _FILE_OFFSET_BITS 64 and plugins/impstats/impstats.c includes config.h I suggest to compile all sources with those flags, which is easiest to do with the "feature=+lfs" option I mentioned at the beginning. I'd prefer an upstream solution in case there is a bug in the upstream build system or the autoconf macro. OpenPGP_signature Description: OpenPGP digital signature
Bug#1032958: rsyslog: Enable Large File Support (LFS)
On 3/14/23 19:06, Helge Deller wrote: That flag is then used only for one C-file in the subdir contrib/omhiredis. Which flag? I meant the compiler preprocessor defines: The c-files should be compiled with: -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE Example: In plugins/impstats/impstats.c the glibc function readdir() is used. Without the "_FILE_OFFSET_BITS=64" define this readdir() will suddenly/randomly fail at runtime if the directory (or the files inside the directory) are stored at an inode with doesn't fit into 32-bits. I suggest to compile all sources with those flags, which is easiest to do with the "feature=+lfs" option I mentioned at the beginning. Helge
Bug#1032958: rsyslog: Enable Large File Support (LFS)
Am 14.03.23 um 18:56 schrieb Helge Deller: On 3/14/23 18:52, Michael Biebl wrote: See also https://github.com/rsyslog/rsyslog/commit/71ffb32ab8a1847f746a298ad20f7dd8b40570a0 I think this patch is wrong. It drops the "define _FILE_OFFSET_BITS 64" from the rsyslog.h file, and instead nearly nowhere adds that flag. The idea afaics is to set that via the global config.h which is included everything. That said, if you think rsyslog is using that macro incorrectly, please raise that upstream at https://github.com/rsyslog/rsyslog/issues Regards, Michael OpenPGP_signature Description: OpenPGP digital signature
Bug#1032958: rsyslog: Enable Large File Support (LFS)
That flag is then used only for one C-file in the subdir contrib/omhiredis. Which flag? I meant the compiler preprocessor defines: The c-files should be compiled with: -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE
Bug#1032958: rsyslog: Enable Large File Support (LFS)
Am 14.03.23 um 18:53 schrieb Helge Deller: On 3/14/23 18:47, Michael Biebl wrote: Control: tags -1 + moreinfo The upstream build system already checks for large file system support via the AC_SYS_LARGEFILE autoconf macro and enables it by default: So for example on i386 I get https://buildd.debian.org/status/fetch.php?pkg=rsyslog=i386=8.2302.0-1=1677101702=0 checking for special C compiler options needed for large files... no checking for _FILE_OFFSET_BITS value needed for large files... 64 ... Large file support enabled: yes Am I missing something? Yes. That flag is then used only for one C-file in the subdir contrib/omhiredis. Which flag? OpenPGP_signature Description: OpenPGP digital signature
Bug#1032958: rsyslog: Enable Large File Support (LFS)
On 3/14/23 18:56, Helge Deller wrote: On 3/14/23 18:52, Michael Biebl wrote: See also https://github.com/rsyslog/rsyslog/commit/71ffb32ab8a1847f746a298ad20f7dd8b40570a0 I think this patch is wrong. It drops the "define _FILE_OFFSET_BITS 64" from the rsyslog.h file, and instead nearly nowhere adds that flag. Please note there is a huge difference between "_LARGE_FILES" and "_FILE_OFFSET_BITS=64". The first allows to read large files, the second allows *access* to (any small and big) files on huge discs. So, this one doesn't report the full truth: #if defined(_LARGE_FILES) || (defined (_FILE_OFFSET_BITS) && _FILE_OFFSET_BITS >= 64) Helge
Bug#1032958: rsyslog: Enable Large File Support (LFS)
On 3/14/23 18:52, Michael Biebl wrote: See also https://github.com/rsyslog/rsyslog/commit/71ffb32ab8a1847f746a298ad20f7dd8b40570a0 I think this patch is wrong. It drops the "define _FILE_OFFSET_BITS 64" from the rsyslog.h file, and instead nearly nowhere adds that flag. Helge
Bug#1032958: rsyslog: Enable Large File Support (LFS)
Am 14.03.23 um 18:47 schrieb Michael Biebl: Control: tags -1 + moreinfo The upstream build system already checks for large file system support via the AC_SYS_LARGEFILE autoconf macro and enables it by default: So for example on i386 I get https://buildd.debian.org/status/fetch.php?pkg=rsyslog=i386=8.2302.0-1=1677101702=0 checking for special C compiler options needed for large files... no checking for _FILE_OFFSET_BITS value needed for large files... 64 ... Large file support enabled: yes Am I missing something? See also https://github.com/rsyslog/rsyslog/commit/71ffb32ab8a1847f746a298ad20f7dd8b40570a0 OpenPGP_signature Description: OpenPGP digital signature
Bug#1032958: rsyslog: Enable Large File Support (LFS)
On 3/14/23 18:47, Michael Biebl wrote: Control: tags -1 + moreinfo The upstream build system already checks for large file system support via the AC_SYS_LARGEFILE autoconf macro and enables it by default: So for example on i386 I get https://buildd.debian.org/status/fetch.php?pkg=rsyslog=i386=8.2302.0-1=1677101702=0 checking for special C compiler options needed for large files... no checking for _FILE_OFFSET_BITS value needed for large files... 64 ... Large file support enabled: yes Am I missing something? Yes. That flag is then used only for one C-file in the subdir contrib/omhiredis. All other occurences of stat() and such are compiled without LFS support. Just search for "FILE_OFFSET_BITS" in the build log you mentioned above. Helge
Bug#1032958: rsyslog: Enable Large File Support (LFS)
Control: tags -1 + moreinfo The upstream build system already checks for large file system support via the AC_SYS_LARGEFILE autoconf macro and enables it by default: So for example on i386 I get https://buildd.debian.org/status/fetch.php?pkg=rsyslog=i386=8.2302.0-1=1677101702=0 checking for special C compiler options needed for large files... no checking for _FILE_OFFSET_BITS value needed for large files... 64 ... Large file support enabled: yes Am I missing something? OpenPGP_signature Description: OpenPGP digital signature
Bug#1032958: rsyslog: Enable Large File Support (LFS)
Package: rsyslog Tags: ftbfs, hppa, patch Version: 8.2302.0-1 rsyslog randomly fails to build from source on the hppa platform due to some testcases randomly failing. I've tracked down various issues, partly related to how the debian buildd server is setup on hppa, but the main problem is, that rsyslog isn't built with large file support, which allows 32-bit platforms (like hppa) to access files on the filesystem beyond the 2GB limit. This affects all "stat()" and "readdir()" glibc calls in the source. If rsyslog is built with LFS enabled (when _FILE_OFFSET_BITS=64 _LARGEFILE_SOURCE are defined for gcc), the testcases succeed. So, please enable LFS for rsyslog in the next upload. The easiest way to do that in a compatible way is to simply add "future=+lfs" to DEB_BUILD_MAINT_OPTIONS in the "debian/rules" file, like this: export DEB_BUILD_MAINT_OPTIONS = hardening=+all future=+lfs This should fix all other 32-bit platforms as well and has no effect on 64-bit platforms. Thanks, Helge
