Your message dated Sat, 28 Sep 2013 23:17:07 +0000
with message-id <e1vq3l5-0001fp...@franck.debian.org>
and subject line Bug#722694: fixed in glusterfs 3.2.7-3+deb7u1
has caused the Debian Bug report #722694,
regarding Kernel change for #685407 breaks GlusterFS when on ext4 bricks
to be marked as done.
This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.
(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact ow...@bugs.debian.org
immediately.)
--
722694: http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=722694
Debian Bug Tracking System
Contact ow...@bugs.debian.org with problems
--- Begin Message ---
Package: glusterfs-server
Version: 3.2.7-3
Severity: grave
Tags: upstream fixed-upstream patch
Control: fixed -1 3.4.0-4
Hi Patrick
Note: I have choosen the severity as 'grave' as it makes GlusterFS not
usable under Debian Wheezy when GlusterFS used on ext4 bricks. I'm
also Cc'ing Jonathan Nieder and Ben Hutchings (at least to have
both informed).
Linux Kernel package linux/3.2.46-1 introduced the following change:
[ Jonathan Nieder ]
* ext3,ext4,nfsd: dir_index: Return 64-bit readdir cookies for NFSv3 and 4
(Closes: #685407)
These are the patches:
# 64-bit NFS readdir cookies on ext3/ext4 with dir_index
bugfix/all/fs-add-new-FMODE-flags-FMODE_32bithash-and-FMODE_64b.patch
bugfix/all/ext4-return-32-64-bit-dir-name-hash-according-to-usa.patch
bugfix/all/nfsd-rename-int-access-to-int-may_flags-in-nfsd_open.patch
bugfix/all/nfsd-vfs_llseek-with-32-or-64-bit-offsets-hashes.patch
bugfix/all/ext3-return-32-64-bit-dir-name-hash-according-to-usa.patch
This kernel change and it's consequences for GlusterFS are mentioned
in [1], where RedHat hat also this issue which reflects in the
bugreport[2].
There is a patch backported to 3.3 which I have applied to test on
3.2.7 in wheezy[3,4,5].
Now after updating the kernel on the servers holding the glusterfs
from linux/3.2.41-2+deb7u2 to linux/3.2.46-1+deb7u1 in consequence I
cannot list anymore the GlusterFS content; a find over it loops
infinitively on getdents system calls and consumes memory.
I have used the attached patch to test, but as it's not my knowledge
aerea, so needs some review if something is missing. I have tested the
following setups:
wheezy + glusterfs/3.2.7-3 + linux/3.2.41-2+deb7u2 -> ok
wheezy + glusterfs/3.2.7-3 + linux/3.2.46-1+deb7u1 -> broken
wheezy + glusterfs/3.2.7-3 + patch + linux/3.2.46-1+deb7u1 -> ok
unstable + glusterfs/3.4.0-4 + linux/3.10.7-1 -> ok
So right now with current given kernel in Wheezy GlusterFS on ext4
bricks is broken, so needs an update trough a stable proposed update
if possible.
[1] http://lwn.net/Articles/544298/
[2] https://bugzilla.redhat.com/show_bug.cgi?id=838784
[3] https://bugzilla.redhat.com/show_bug.cgi?id=838784#c24
[4] http://review.gluster.org/#/c/4822/
[5]
http://review.gluster.org/#/c/4822/2/xlators/cluster/dht/src/dht-helper.c,unified
Regards,
Salvatore
--- a/xlators/cluster/dht/src/dht-helper.c
+++ b/xlators/cluster/dht/src/dht-helper.c
@@ -49,6 +40,43 @@
}
+static uint64_t
+dht_bits_for (uint64_t num)
+{
+ uint64_t bits = 0, ctrl = 1;
+
+ while (ctrl < num) {
+ ctrl *= 2;
+ bits ++;
+ }
+
+ return bits;
+}
+
+/*
+ * A slightly "updated" version of the algorithm described in the commit log
+ * is used here.
+ *
+ * The only enhancement is that:
+ *
+ * - The number of bits used by the backend filesystem for HUGE d_off which
+ * is described as 63, and
+ * - The number of bits used by the d_off presented by the transformation
+ * upwards which is described as 64, are both made "configurable."
+ */
+
+
+#define BACKEND_D_OFF_BITS 63
+#define PRESENT_D_OFF_BITS 63
+
+#define ONE 1ULL
+#define MASK (~0ULL)
+#define PRESENT_MASK (MASK >> (64 - PRESENT_D_OFF_BITS))
+#define BACKEND_MASK (MASK >> (64 - BACKEND_D_OFF_BITS))
+
+#define TOP_BIT (ONE << (PRESENT_D_OFF_BITS - 1))
+#define SHIFT_BITS (max (0, (BACKEND_D_OFF_BITS - PRESENT_D_OFF_BITS + 1)))
+
int
dht_itransform (xlator_t *this, xlator_t *subvol, uint64_t x, uint64_t *y_p)
{
@@ -56,6 +84,9 @@
int cnt = 0;
int max = 0;
uint64_t y = 0;
+ uint64_t hi_mask = 0;
+ uint64_t off_mask = 0;
+ int max_bits = 0;
if (x == ((uint64_t) -1)) {
y = (uint64_t) -1;
@@ -69,7 +100,23 @@
max = conf->subvolume_cnt;
cnt = dht_subvol_cnt (this, subvol);
- y = ((x * max) + cnt);
+ if (max == 1) {
+ y = x;
+ goto out;
+ }
+
+ max_bits = dht_bits_for (max);
+
+ hi_mask = ~(PRESENT_MASK >> (max_bits + 1));
+
+ if (x & hi_mask) {
+ /* HUGE d_off */
+ off_mask = MASK << max_bits;
+ y = TOP_BIT | ((x >> SHIFT_BITS) & off_mask) | cnt;
+ } else {
+ /* small d_off */
+ y = ((x * max) + cnt);
+ }
out:
if (y_p)
@@ -147,16 +193,38 @@
int max = 0;
uint64_t x = 0;
xlator_t *subvol = 0;
+ int max_bits = 0;
+ uint64_t off_mask = 0;
+ uint64_t host_mask = 0;
if (!this->private)
- goto out;
+ return -1;
conf = this->private;
max = conf->subvolume_cnt;
- cnt = y % max;
- x = y / max;
+ if (max == 1) {
+ x = y;
+ cnt = 0;
+ goto out;
+ }
+
+ if (y & TOP_BIT) {
+ /* HUGE d_off */
+ max_bits = dht_bits_for (max);
+ off_mask = (MASK << max_bits);
+ host_mask = ~(off_mask);
+
+ x = ((y & ~TOP_BIT) & off_mask) << SHIFT_BITS;
+
+ cnt = y & host_mask;
+ } else {
+ /* small d_off */
+ cnt = y % max;
+ x = y / max;
+ }
+out:
subvol = conf->subvolumes[cnt];
if (subvol_p)
@@ -165,7 +233,6 @@
if (x_p)
*x_p = x;
-out:
return 0;
}
signature.asc
Description: Digital signature
--- End Message ---
--- Begin Message ---
Source: glusterfs
Source-Version: 3.2.7-3+deb7u1
We believe that the bug you reported is fixed in the latest version of
glusterfs, which is due to be installed in the Debian FTP archive.
A summary of the changes between this version and the previous one is
attached.
Thank you for reporting the bug, which will now be closed. If you
have further comments please address them to 722...@bugs.debian.org,
and the maintainer will reopen the bug report if appropriate.
Debian distribution maintenance software
pp.
Patrick Matthäi <pmatth...@debian.org> (supplier of updated glusterfs package)
(This message was generated automatically at their request; if you
believe that there is a problem with it please contact the archive
administrators by mailing ftpmas...@ftp-master.debian.org)
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Format: 1.8
Date: Fri, 27 Sep 2013 21:13:42 +0200
Source: glusterfs
Binary: glusterfs-client glusterfs-server glusterfs-examples glusterfs-common
glusterfs-dbg
Architecture: source all amd64
Version: 3.2.7-3+deb7u1
Distribution: stable
Urgency: low
Maintainer: Patrick Matthäi <pmatth...@debian.org>
Changed-By: Patrick Matthäi <pmatth...@debian.org>
Description:
glusterfs-client - clustered file-system (client package)
glusterfs-common - GlusterFS common libraries and translator modules
glusterfs-dbg - GlusterFS debugging symbols
glusterfs-examples - example files for the glusterfs server and client
glusterfs-server - clustered file-system (server package)
Closes: 722694
Changes:
glusterfs (3.2.7-3+deb7u1) stable; urgency=low
.
* Add patch 04-ext4-safe to fix broken glusterfs with Linux
>= 3.2.46-1+deb7u1 if you use ext4 as storage backend.
Closes: #722694
Checksums-Sha1:
b1c55525360132708d872057cbf8f5336d156af9 2252 glusterfs_3.2.7-3+deb7u1.dsc
2de704997f21458e8e07b8e39d5262a6a5f10fef 15735
glusterfs_3.2.7-3+deb7u1.debian.tar.gz
a8a8178b7cd01af873287caf1230801937ca9096 384722
glusterfs-examples_3.2.7-3+deb7u1_all.deb
1badfb7a12f7947d0d7e35889136ac20282993ec 384998
glusterfs-client_3.2.7-3+deb7u1_amd64.deb
500eb6a96ed8608e51f72fee7ad66f229da57300 442782
glusterfs-server_3.2.7-3+deb7u1_amd64.deb
1180bc06f4c2a4b8e3e748085530b5c1eaf01849 11008394
glusterfs-common_3.2.7-3+deb7u1_amd64.deb
97fb5f9a2736d2eb540dc743684b743be19c4524 5788700
glusterfs-dbg_3.2.7-3+deb7u1_amd64.deb
Checksums-Sha256:
7052c678ee023e04b009edd37616bdfdb51caf02f327ed4c2ceb73e706c95afd 2252
glusterfs_3.2.7-3+deb7u1.dsc
67dc6cd7e3f4583b56218c8c89317f369c7f9210328c6f99d4d5ea8c85c7d4fc 15735
glusterfs_3.2.7-3+deb7u1.debian.tar.gz
67b466895d58ad7bc7c18933c468da9e5802b1365c38c2e747d5d9a7a1d97a22 384722
glusterfs-examples_3.2.7-3+deb7u1_all.deb
3ba2e003ffa921803951f988c97368062ebe02d0ae2fab992ef6b41e365beb61 384998
glusterfs-client_3.2.7-3+deb7u1_amd64.deb
865e221f1a8bf3eb9cd68a1e29a6644809f49156d884f95c8ca4b714aa580a68 442782
glusterfs-server_3.2.7-3+deb7u1_amd64.deb
27f344636e42f327d1b71334a5e3724fdc98b29c5dc1dc7be1974a154500aed1 11008394
glusterfs-common_3.2.7-3+deb7u1_amd64.deb
17359d935f5ef9482286d36ccd947b2bb12e19501d42f362e963e218d15a38f3 5788700
glusterfs-dbg_3.2.7-3+deb7u1_amd64.deb
Files:
4cae531431dd1820a4f9022ae4223806 2252 admin optional
glusterfs_3.2.7-3+deb7u1.dsc
c622fb48307d7e4209a529e0d89ca604 15735 admin optional
glusterfs_3.2.7-3+deb7u1.debian.tar.gz
3e71f4715d6739c9b1a167f1e6a161ad 384722 admin optional
glusterfs-examples_3.2.7-3+deb7u1_all.deb
349e9de8a1bbb53c4f73ab6635102b50 384998 admin optional
glusterfs-client_3.2.7-3+deb7u1_amd64.deb
fe9e49f38556fcb616420c82b12f7993 442782 admin optional
glusterfs-server_3.2.7-3+deb7u1_amd64.deb
fa0fed95ac6281ac31afaf5f11123ca5 11008394 libs optional
glusterfs-common_3.2.7-3+deb7u1_amd64.deb
62c45bfd0d8ab14ff80517272292bc56 5788700 debug extra
glusterfs-dbg_3.2.7-3+deb7u1_amd64.deb
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.12 (GNU/Linux)
iQIcBAEBCAAGBQJSRx2JAAoJEBLZsEqQy9jkpMYQAJtSoR5Jy9MPNrQ48nlp+huG
gxuH/qwTdCku1vUTfjhAiQdPT1q9wuZaBBZzMnGZ1gVUliW0fixpFHwq3Jx6X7Y2
3tv0Aa2R3XVOV6HTitjBHkjzQAm0TR/Rkw/yqOoxhFxB0zKyy/5vpZB57gLq1VBw
HX0flhrohhNzLO895wUUq6HjKoiRuFGTuOpCQrz5oPZqrrgZQML1kTXOgPT9ZgJB
tjSVzaKQv3yXQZLVw7yBvWQkUKjRAzonQRqGdFgoUuDKDLBJYiMkuhX+PorwVXdb
RbFGxrI9kfuvRNDjnJpYC3Ws2PbAdnlc5qEXqELkm9wsfSJbOaEOSEcQoqxXioNa
/v7dfUqsRWaPPCiJqJZ+xson1edryELlo3rGbYr7yRLNBDwSrt0FU4nXKe0/PVca
MmqE15ZGxomBALfRYFBfwF4k/r1pOhwiCxXPDoxKsBAHuXmhKXBhcwEIdDoH/n0d
0C7BlsEf3nYTkY7VnCA9WZIlTp7r54VEFm3UGbMF+O2QBPJUt944P326yHxHQegD
cdCiasqDSSKK7mmLomh0tv5b2ekvSnMMk3b8XAdW0MZMZYCXI/xwstIkV74MW1v1
EcBg41pv3I169fEnmPsk0VQZoCXxWVCk+pCTT82kEB/drxOUsqHjv5EAaXZ0sOGA
9ARXhanrmyBBdka1QhrD
=k++O
-----END PGP SIGNATURE-----
--- End Message ---
