Bug#590513: apt-get fails on i386 with 20tb filesystem (statvfs (75: Value too large for defined data type))
On Do, 2010-08-26 at 03:59 +0200, Guillem Jover wrote:
Hi!
On Tue, 2010-07-27 at 13:29:07 +0200, Julian Andres Klode wrote:
On Mo, 2010-07-26 at 15:23 -0700, David Bronaugh wrote:
When I try to install a package inside an i386 chroot on this machine
(which resides on a 20TB XFS filesystem), it fails to even get to
downloading the package with the following error messages:
It seems like this is likely a result of integer overflow in the data
structure that statvfs returns. Therefore, it would seem the path
forward is to use statvfs64().
Fixed in bzr:
=== modified file 'cmdline/apt-get.cc'
--- cmdline/apt-get.cc 2010-07-05 09:34:35 +
+++ cmdline/apt-get.cc 2010-07-27 11:21:03 +
@@ -25,6 +25,9 @@
# */
/*}}}*/
// Include Files /*{{{*/
+#define _LARGEFILE_SOURCE
+#define _LARGEFILE64_SOURCE
+
#include apt-pkg/error.h
#include apt-pkg/cmndline.h
#include apt-pkg/init.h
@@ -63,6 +66,9 @@
#include regex.h
#include sys/wait.h
#include sstream
+
+#define statfs statfs64
+#define statvfs statvfs64
/*}}}*/
#define RAMFS_MAGIC 0x858458f6
There's few problems with this patch. By defining _LARGEFILE_SOURCE,
the source is already requesting to the system to internally remap
statfs and statvfs to statfs64 and statvfs64. And in addition
_LARGEFILE64_SOURCE just makes the explicit 64 bit versions visible,
at which point the system was alredy transparently supporting large
files.
No, defining _FILE_OFFSET_BITS to 64 makes stat* become stat*64. The
_LARGEFILE_SOURCE and _LARGEFILE64_SOURCE defines are needed for getting
the explicit interfaces, they do not activate the implicit ones.
It unconditionally remaps itself the symbols to their 64 bit versions,
but they might not be available on the system, the correct way to check
for this is by using AC_SYS_LARGEFILE in configure.in.
Maybe in theory; in practice, it works perfectly for all our supported
architectures.
It also does not fix the casted comparisons for some of the statfs and
statvfs members which will be now 64 bit wide on 32 bit systems.
I did not find such problems when looking at it.
--
Julian Andres Klode - Debian Developer, Ubuntu Member
See http://wiki.debian.org/JulianAndresKlode and http://jak-linux.org/.
--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#590513: apt-get fails on i386 with 20tb filesystem (statvfs (75: Value too large for defined data type))
On Thu, 2010-08-26 at 08:40:35 +0200, Julian Andres Klode wrote: On Do, 2010-08-26 at 03:59 +0200, Guillem Jover wrote: There's few problems with this patch. By defining _LARGEFILE_SOURCE, the source is already requesting to the system to internally remap statfs and statvfs to statfs64 and statvfs64. And in addition _LARGEFILE64_SOURCE just makes the explicit 64 bit versions visible, at which point the system was alredy transparently supporting large files. No, defining _FILE_OFFSET_BITS to 64 makes stat* become stat*64. The _LARGEFILE_SOURCE and _LARGEFILE64_SOURCE defines are needed for getting the explicit interfaces, they do not activate the implicit ones. Bah, yeah sorry, meant that one. It unconditionally remaps itself the symbols to their 64 bit versions, but they might not be available on the system, the correct way to check for this is by using AC_SYS_LARGEFILE in configure.in. Maybe in theory; in practice, it works perfectly for all our supported architectures. Well, apt is not only being used on Debian systems, and doing the right thing here (portability-wise) does not require much effort anyway. As said, if using AC_SYS_LARGEFILE is deemed too dangerious at this point in time then defining _FILE_OFFSET_BITS=64 and fixing the casts is the only thing needed to make the code always DTRT, even if the system does not have LFS. It also does not fix the casted comparisons for some of the statfs and statvfs members which will be now 64 bit wide on 32 bit systems. I did not find such problems when looking at it. ,--- cmdline/apt-get.cc:InstallPackages() [...] } else if (unsigned(Buf.f_bfree) (FetchBytes - FetchPBytes)/Buf.f_bsize) } [...] return _error-Error(_(You don't have enough free space in %s.), OutputDir.c_str()); } [...] `--- The unsigned() cast will truncate the 64 bit value, which will make the program fail whenever the lower bits are than the needed size. regards, guillem -- To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#590513: apt-get fails on i386 with 20tb filesystem (statvfs (75: Value too large for defined data type))
Hi!
On Tue, 2010-07-27 at 13:29:07 +0200, Julian Andres Klode wrote:
On Mo, 2010-07-26 at 15:23 -0700, David Bronaugh wrote:
When I try to install a package inside an i386 chroot on this machine
(which resides on a 20TB XFS filesystem), it fails to even get to
downloading the package with the following error messages:
It seems like this is likely a result of integer overflow in the data
structure that statvfs returns. Therefore, it would seem the path
forward is to use statvfs64().
Fixed in bzr:
=== modified file 'cmdline/apt-get.cc'
--- cmdline/apt-get.cc2010-07-05 09:34:35 +
+++ cmdline/apt-get.cc2010-07-27 11:21:03 +
@@ -25,6 +25,9 @@
# */
/*}}}*/
// Include Files /*{{{*/
+#define _LARGEFILE_SOURCE
+#define _LARGEFILE64_SOURCE
+
#include apt-pkg/error.h
#include apt-pkg/cmndline.h
#include apt-pkg/init.h
@@ -63,6 +66,9 @@
#include regex.h
#include sys/wait.h
#include sstream
+
+#define statfs statfs64
+#define statvfs statvfs64
/*}}}*/
#define RAMFS_MAGIC 0x858458f6
There's few problems with this patch. By defining _LARGEFILE_SOURCE,
the source is already requesting to the system to internally remap
statfs and statvfs to statfs64 and statvfs64. And in addition
_LARGEFILE64_SOURCE just makes the explicit 64 bit versions visible,
at which point the system was alredy transparently supporting large
files.
It unconditionally remaps itself the symbols to their 64 bit versions,
but they might not be available on the system, the correct way to check
for this is by using AC_SYS_LARGEFILE in configure.in.
It also does not fix the casted comparisons for some of the statfs and
statvfs members which will be now 64 bit wide on 32 bit systems.
As the API does not seem to expose any off_t type publicly, it seems
safe to enable large file support globally, if there's some exposed
API which could be affected, then just defining _LARGEFILE_SOURCE for
now at the top of this source and not doing the manual remapping would
do too. And the global switch can be done later on.
thanks,
guillem
--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#590513: apt-get fails on i386 with 20tb filesystem (statvfs (75: Value too large for defined data type))
On Mo, 2010-07-26 at 15:23 -0700, David Bronaugh wrote:
Package: apt
Version: 0.7.25.3
When I try to install a package inside an i386 chroot on this machine
(which resides on a 20TB XFS filesystem), it fails to even get to
downloading the package with the following error messages:
r...@windy:~# apt-get install tinywm
Reading package lists... Done
Building dependency tree
Reading state information... Done
The following extra packages will be installed:
libx11-6 libx11-data libxau6 libxcb1 libxdmcp6
The following NEW packages will be installed:
libx11-6 libx11-data libxau6 libxcb1 libxdmcp6 tinywm
0 upgraded, 6 newly installed, 0 to remove and 6 not upgraded.
Need to get 1131kB of archives.
After this operation, 4346kB of additional disk space will be used.
W: Couldn't determine free space in /var/cache/apt/archives/ - statvfs
(75: Value too large for defined data type)
E: Handler silently failed
r...@windy:~#
It seems like this is likely a result of integer overflow in the data
structure that statvfs returns. Therefore, it would seem the path
forward is to use statvfs64().
Fixed in bzr:
=== modified file 'cmdline/apt-get.cc'
--- cmdline/apt-get.cc 2010-07-05 09:34:35 +
+++ cmdline/apt-get.cc 2010-07-27 11:21:03 +
@@ -25,6 +25,9 @@
# */
/*}}}*/
// Include Files /*{{{*/
+#define _LARGEFILE_SOURCE
+#define _LARGEFILE64_SOURCE
+
#include apt-pkg/error.h
#include apt-pkg/cmndline.h
#include apt-pkg/init.h
@@ -63,6 +66,9 @@
#include regex.h
#include sys/wait.h
#include sstream
+
+#define statfs statfs64
+#define statvfs statvfs64
/*}}}*/
#define RAMFS_MAGIC 0x858458f6
=== modified file 'debian/changelog'
--- debian/changelog2010-07-23 14:13:15 +
+++ debian/changelog2010-07-27 11:27:18 +
@@ -5,6 +5,9 @@ apt (0.7.26~exp11) experimental; urgency
- Add to history whether a change was automatic or not.
* apt-pkg/contrib/fileutl.cc:
- Add FileFd::OpenDescriptor() (needed for python-apt's #383617).
+ * cmdline/apt-get.cc:
+- Support large filesystems by using statvfs64() instead of statvfs()
+ and statfs64() instead of statfs() (Closes: #590513).
-- Julian Andres Klode j...@debian.org Wed, 21 Jul 2010 17:09:11 +0200
--
Julian Andres Klode - Debian Developer, Ubuntu Member
See http://wiki.debian.org/JulianAndresKlode and http://jak-linux.org/.
--
To UNSUBSCRIBE, email to debian-bugs-dist-requ...@lists.debian.org
with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org
Bug#590513: apt-get fails on i386 with 20tb filesystem (statvfs (75: Value too large for defined data type))
Package: apt Version: 0.7.25.3 When I try to install a package inside an i386 chroot on this machine (which resides on a 20TB XFS filesystem), it fails to even get to downloading the package with the following error messages: r...@windy:~# apt-get install tinywm Reading package lists... Done Building dependency tree Reading state information... Done The following extra packages will be installed: libx11-6 libx11-data libxau6 libxcb1 libxdmcp6 The following NEW packages will be installed: libx11-6 libx11-data libxau6 libxcb1 libxdmcp6 tinywm 0 upgraded, 6 newly installed, 0 to remove and 6 not upgraded. Need to get 1131kB of archives. After this operation, 4346kB of additional disk space will be used. W: Couldn't determine free space in /var/cache/apt/archives/ - statvfs (75: Value too large for defined data type) E: Handler silently failed r...@windy:~# It seems like this is likely a result of integer overflow in the data structure that statvfs returns. Therefore, it would seem the path forward is to use statvfs64().
