Re: Reiser4 und LZO compression
On Tue, Aug 29, 2006 at 07:48:25AM +1000, Nigel Cunningham wrote: For Suspend2, we ended up converting the LZF support to a cryptoapi plugin. Is there any chance that you could use cryptoapi modules? We could then have a hope of sharing the support. Using cryptoapi plugins for the compression methods is an interesting approach, there's a few other places in the kernel that could probably benefit from this as well, such as jffs2 (which at the moment rolls its own compression subsystem), and the out-of-tree page and swap cache compression work. Assuming you were wrapping in to LZF directly prior to the cryptoapi integration, do you happen to have before and after numbers to determine how heavyweight the rest of the cryptoapi overhead is? It would be interesting to profile this and consider migrating the in-tree users, rather than duplicating the compress/decompress routines all over the place.
Re: Reiser4 und LZO compression
Hi. On Tue, 2006-08-29 at 06:05 +0200, Jan Engelhardt wrote: Hmm. LZO is the best compression algorithm for the task as measured by the objectives of good compression effectiveness while still having very low CPU usage (the best of those written and GPL'd, there is a slightly better one which is proprietary and uses more CPU, LZRW if I remember right. The gzip code base uses too much CPU, though I think Edward made I don't think that LZO beats LZF in both speed and compression ratio. LZF is also available under GPL (dual-licensed BSD) and was choosen in favor of LZO for the next generation suspend-to-disk code of the Linux kernel. see: http://www.goof.com/pcg/marc/liblzf.html thanks for the info, we will compare them For Suspend2, we ended up converting the LZF support to a cryptoapi plugin. Is there any chance that you could use cryptoapi modules? We could then have a hope of sharing the support. I am throwing in gzip: would it be meaningful to use that instead? The decoder (inflate.c) is already there. 06:04 shanghai:~/liblzf-1.6 l configure* -rwxr-xr-x 1 jengelh users 154894 Mar 3 2005 configure -rwxr-xr-x 1 jengelh users 26810 Mar 3 2005 configure.bz2 -rw-r--r-- 1 jengelh users 30611 Aug 28 20:32 configure.gz-z9 -rw-r--r-- 1 jengelh users 30693 Aug 28 20:32 configure.gz-z6 -rw-r--r-- 1 jengelh users 53077 Aug 28 20:32 configure.lzf We used gzip when we first implemented compression support, and found it to be far too slow. Even with the fastest compression options, we were only getting a few megabytes per second. Perhaps I did something wrong in configuring it, but there's not that many things to get wrong! In contrast, with LZF, we get very high throughput. My current laptop is an 1.8MHz Turion with a 7200 RPM (PATA) drive. Without LZF compression, my throughput in writing an image is the maximum the drive interface can manage - 38MB/s. With LZF, I get roughly that divided by compression ratio achieved, so if the compression ratio is ~50%, as it generally is, I'm reading and writing the image at 75-80MB/s. During this time, all the computer is doing is compressing pages using LZF and submitting bios, with the odd message being send to the userspace interface app via netlink. I realise this is very different to the workload you'll be doing, but hopefully the numbers are somewhat useful: [EMAIL PROTECTED]:~$ cat /sys/power/suspend2/debug_info Suspend2 debugging info: - SUSPEND core : 2.2.7.4 - Kernel Version : 2.6.18-rc4 - Compiler vers. : 4.1 - Attempt number : 1 - Parameters : 0 32785 0 0 0 0 - Overall expected compression percentage: 0. - Compressor is 'lzf'. Compressed 820006912 bytes into 430426371 (47 percent compression). - Swapwriter active. Swap available for image: 487964 pages. - Filewriter inactive. - I/O speed: Write 74 MB/s, Read 70 MB/s. - Extra pages: 1913 used/2100. [EMAIL PROTECTED]:~$ (Modify hibernate.conf to disable compression, suspend again...) [EMAIL PROTECTED]:~$ cat /sys/power/suspend2/debug_info Suspend2 debugging info: - SUSPEND core : 2.2.7.4 - Kernel Version : 2.6.18-rc4 - Compiler vers. : 4.1 - Attempt number : 2 - Parameters : 0 32785 0 0 0 0 - Overall expected compression percentage: 0. - Swapwriter active. Swap available for image: 487964 pages. - Filewriter inactive. - I/O speed: Write 38 MB/s, Read 39 MB/s. - Extra pages: 1907 used/2100. [EMAIL PROTECTED]:~$ Oh, I also have a debugging mode where I can get Suspend2 to just compress the pages but not actually write anything. If I do that, it says it can do 80MB/s on my kernel image, so the disk is still the bottleneck, it seems. Hope this all helps (and isn't information overload!) Nigel
Re: Reiser4 und LZO compression
Hi. On Tue, 2006-08-29 at 13:59 +0900, Paul Mundt wrote: On Tue, Aug 29, 2006 at 07:48:25AM +1000, Nigel Cunningham wrote: For Suspend2, we ended up converting the LZF support to a cryptoapi plugin. Is there any chance that you could use cryptoapi modules? We could then have a hope of sharing the support. Using cryptoapi plugins for the compression methods is an interesting approach, there's a few other places in the kernel that could probably benefit from this as well, such as jffs2 (which at the moment rolls its own compression subsystem), and the out-of-tree page and swap cache compression work. Assuming you were wrapping in to LZF directly prior to the cryptoapi integration, do you happen to have before and after numbers to determine how heavyweight the rest of the cryptoapi overhead is? It would be interesting to profile this and consider migrating the in-tree users, rather than duplicating the compress/decompress routines all over the place. I was, but I don't have numbers right now. I'm about to go out, but will see if I can find them when I get back later. From memory, it wasn't a huge change in terms of lines of code. Regards, Nigel
Re: Reiser4 und LZO compression
On Sun, 27 Aug 2006 04:34:26 +0400 Alexey Dobriyan [EMAIL PROTECTED] wrote: The patch below is so-called reiser4 LZO compression plugin as extracted from 2.6.18-rc4-mm3. I think it is an unauditable piece of shit and thus should not enter mainline. Like lib/inflate.c (and this new code should arguably be in lib/). The problem is that if we clean this up, we've diverged very much from the upstream implementation. So taking in fixes and features from upstream becomes harder and more error-prone. I'd suspect that the maturity of these utilities is such that we could afford to turn them into kernel code in the expectation that any future changes will be small. But it's not a completely simple call. (iirc the inflate code had a buffer overrun a while back, which was found and fixed in the upstream version).
Re: Reiser4 und LZO compression
On 8/27/06, Andrew Morton [EMAIL PROTECTED] wrote: On Sun, 27 Aug 2006 04:34:26 +0400 Alexey Dobriyan [EMAIL PROTECTED] wrote: The patch below is so-called reiser4 LZO compression plugin as extracted from 2.6.18-rc4-mm3. I think it is an unauditable piece of shit and thus should not enter mainline. Sheesh. Like lib/inflate.c (and this new code should arguably be in lib/). The problem is that if we clean this up, we've diverged very much from the upstream implementation. So taking in fixes and features from upstream becomes harder and more error-prone. Right. How about putting it in as so that everyone can track divergences, but to not use it for a real compile. Rather, consider it meta-source, and do mechanical, repeatable transformations only, starting with something like: mv minilzo.c minilzo._c cpp 2/dev/null -w -P -C -nostdinc -dI minilzo._c minilzo.c lindent minilzo.c to generate a version that can be audited. Doing so on a version of minilzo.c google found on the web generated something that looked much like any other stream coder source I've read, so it approaches readability. Of a sorts. Further cleanups could be done with cpp -D to rename some of the more bizarre symbols. Downside is that bugs would have to be fixed in the 'meta-source' (horrible name, but it's late here), but at least they could be found (potentially) easier than in the original. Ray
Re: Reiser4 und LZO compression
Andrew Morton wrote: On Sun, 27 Aug 2006 04:34:26 +0400 Alexey Dobriyan [EMAIL PROTECTED] wrote: The patch below is so-called reiser4 LZO compression plugin as extracted from 2.6.18-rc4-mm3. I think it is an unauditable piece of shit and thus should not enter mainline. Like lib/inflate.c (and this new code should arguably be in lib/). The problem is that if we clean this up, we've diverged very much from the upstream implementation. So taking in fixes and features from upstream becomes harder and more error-prone. Well, what kinds of changes have to happen? I doubt upstream would care about moving some of it to lib/ -- and anyway, reiserfs-list is on the CC. We are speaking of upstream in the third party in the presence of upstream, so... Maybe just ask upstream?
Re: reiser4-2.6.18-rc2-mm1: possible circular locking dependency detected in txn_end
Le 22.08.2006 15:08, Alexander Zarochentsev a écrit :
Hello,
On 12 August 2006 17:26, Laurent Riffard wrote:
Le 03.08.2006 17:07, Laurent Riffard a écrit :
Le 03.08.2006 08:09, Alexander Zarochentsev a écrit :
On Tuesday 01 August 2006 01:29, Laurent Riffard wrote:
Le 31.07.2006 21:55, Vladimir V. Saveliev a écrit :
Hello
What kind of load did you run on reiser4 at that time?
I just formatted a new 2GB Reiser4 FS, then I moved a whole
ccache cache tree to this new FS (cache size was about 20~30
Mbytes). Something like:
# mkfs.reiser4 /dev/vglinux1/ccache
# mount -tauto -onoatime /dev/vglinux1/ccache /mnt/disk
# mv ~laurent/.ccache/* /mnt/disk/
I was not able to reproduce it. Can you please try the following
patch?
lock validator friendly locking of new atom in
atom_begin_and_assign_to_txnh and locking of two atoms.
Signed-off-by: Alexander Zarochentsev [EMAIL PROTECTED]
---
fs/reiser4/txnmgr.c | 14 --
fs/reiser4/txnmgr.h | 15 +++
2 files changed, 23 insertions(+), 6 deletions(-)
[patch snipped]
I tried this patch: it's slow as hell (CPU is ~100% system) and it
overhead of locking dependency checks?
also disabling CONFIG_REISER4_DEBUG should help tp reduce cpu usage.
panics when syncing...
please apply another patch
Sorry for the late answer, I had problems with my ISP...
I applied the below patch on kernel 2.6.18-rc2-mm2: the lock validator
warning does not happen anymore (it did happen with plain 2.6.18-rc2-mm2)
and I wasn't able trigger a reiser4 panic.
The bad news is that it's still very slow, eating a lot of system CPU (Yes, I
disabled CONFIG_REISER4_DEBUG).
Here are some real life numbers: I copied my kernel tree on 2 different FS
and I timed the compilation of a single directory. First FS is reiserfs, the
other is reiser4.
# mount | grep kernel
/dev/mapper/vglinux1-lvkernel on /home/laurent/kernel type reiserfs (rw,noatime)
/dev/hda5 on /home/laurent/kernel2 type reiser4 (rw)
# df | grep kernel
/dev/mapper/vglinux1-lvkernel
1,1G 776M 257M 76% /home/laurent/kernel
/dev/hda5 2,0G 720M 1,3G 36% /home/laurent/kernel2
The scenario is:
$ cd ~/kernel/linux-2.6-mm
$ # clean fs/reiser4/
$ find fs/reiser4/ -name *.o -type f| xargs rm
$ sync
$ time make fs/reiser4/
Reiserfs, 2.6.18-rc2-mm2:
57.65user 8.66system 1:12.32elapsed 91%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+599977minor)pagefaults 0swaps
Reiser4, plain 2.6.18-rc2-mm2:
57.85user 22.89system 1:27.01elapsed 92%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+600928minor)pagefaults 0swaps
Reiser4, 2.6.18-rc2-mm2 patched:
59.26user 301.93system 7:00.99elapsed 85%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+601058minor)pagefaults 0swaps
~~
laurent
lock validator friendly locking of new atom in
atom_begin_and_assign_to_txnh and locking of two atoms.
Signed-off-by: Alexander Zarochentsev [EMAIL PROTECTED]
---
fs/reiser4/txnmgr.c | 14 --
fs/reiser4/txnmgr.h | 15 +++
2 files changed, 23 insertions(+), 6 deletions(-)
--- linux-2.6-git.orig/fs/reiser4/txnmgr.c
+++ linux-2.6-git/fs/reiser4/txnmgr.c
@@ -397,7 +397,7 @@ static void atom_init(txn_atom * atom)
INIT_LIST_HEAD(ATOM_OVRWR_LIST(atom));
INIT_LIST_HEAD(ATOM_WB_LIST(atom));
INIT_LIST_HEAD(atom-inodes);
- spin_lock_init(atom-alock);
+ spin_lock_init((atom-alock));
/* list of transaction handles */
INIT_LIST_HEAD(atom-txnh_list);
/* link to transaction manager's list of atoms */
@@ -732,10 +732,12 @@ static int atom_begin_and_assign_to_txnh
assert(jmacd-17, atom_isclean(atom));
/*
- * do not use spin_lock_atom because we have broken lock ordering here
- * which is ok, as long as @atom is new and inaccessible for others.
+ * lock ordering is broken here. It is ok, as long as @atom is new
+ * and inaccessible for others. We can't use spin_lock_atom or
+ * spin_lock(atom-alock) because they care about locking
+ * dependencies. spin_trylock_lock doesn't.
*/
- spin_lock((atom-alock));
+ check_me(, spin_trylock_atom(atom));
/* add atom to the end of transaction manager's list of atoms */
list_add_tail(atom-atom_link, mgr-atoms_list);
@@ -751,7 +753,7 @@ static int atom_begin_and_assign_to_txnh
atom-super = reiser4_get_current_sb();
capture_assign_txnh_nolock(atom, txnh);
- spin_unlock((atom-alock));
+ spin_unlock_atom(atom);
spin_unlock_txnh(txnh);
return -E_REPEAT;
@@ -2112,11 +2114,11 @@ static void fuse_not_fused_lock_owners(t
atomic_inc(atomf-refcount);
spin_unlock_txnh(ctx-trans);
if (atomf atomh) {
- spin_lock_atom(atomf);
+ spin_lock_atom_nested(atomf);
} else {
Re: reiser4-2.6.18-rc2-mm1: possible circular locking dependency detected in txn_end
Hello,
On 12 August 2006 17:26, Laurent Riffard wrote:
Le 03.08.2006 17:07, Laurent Riffard a écrit :
Le 03.08.2006 08:09, Alexander Zarochentsev a écrit :
On Tuesday 01 August 2006 01:29, Laurent Riffard wrote:
Le 31.07.2006 21:55, Vladimir V. Saveliev a écrit :
Hello
What kind of load did you run on reiser4 at that time?
I just formatted a new 2GB Reiser4 FS, then I moved a whole
ccache cache tree to this new FS (cache size was about 20~30
Mbytes). Something like:
# mkfs.reiser4 /dev/vglinux1/ccache
# mount -tauto -onoatime /dev/vglinux1/ccache /mnt/disk
# mv ~laurent/.ccache/* /mnt/disk/
I was not able to reproduce it. Can you please try the following
patch?
lock validator friendly locking of new atom in
atom_begin_and_assign_to_txnh and locking of two atoms.
Signed-off-by: Alexander Zarochentsev [EMAIL PROTECTED]
---
fs/reiser4/txnmgr.c | 14 --
fs/reiser4/txnmgr.h | 15 +++
2 files changed, 23 insertions(+), 6 deletions(-)
[patch snipped]
I tried this patch: it's slow as hell (CPU is ~100% system) and it
overhead of locking dependency checks?
also disabling CONFIG_REISER4_DEBUG should help tp reduce cpu usage.
panics when syncing...
please apply another patch
lock validator friendly locking of new atom in
atom_begin_and_assign_to_txnh and locking of two atoms.
Signed-off-by: Alexander Zarochentsev [EMAIL PROTECTED]
---
fs/reiser4/txnmgr.c | 14 --
fs/reiser4/txnmgr.h | 15 +++
2 files changed, 23 insertions(+), 6 deletions(-)
--- linux-2.6-git.orig/fs/reiser4/txnmgr.c
+++ linux-2.6-git/fs/reiser4/txnmgr.c
@@ -397,7 +397,7 @@ static void atom_init(txn_atom * atom)
INIT_LIST_HEAD(ATOM_OVRWR_LIST(atom));
INIT_LIST_HEAD(ATOM_WB_LIST(atom));
INIT_LIST_HEAD(atom-inodes);
- spin_lock_init(atom-alock);
+ spin_lock_init((atom-alock));
/* list of transaction handles */
INIT_LIST_HEAD(atom-txnh_list);
/* link to transaction manager's list of atoms */
@@ -732,10 +732,12 @@ static int atom_begin_and_assign_to_txnh
assert(jmacd-17, atom_isclean(atom));
/*
-* do not use spin_lock_atom because we have broken lock ordering here
-* which is ok, as long as @atom is new and inaccessible for others.
+* lock ordering is broken here. It is ok, as long as @atom is new
+* and inaccessible for others. We can't use spin_lock_atom or
+* spin_lock(atom-alock) because they care about locking
+* dependencies. spin_trylock_lock doesn't.
*/
- spin_lock((atom-alock));
+ check_me(, spin_trylock_atom(atom));
/* add atom to the end of transaction manager's list of atoms */
list_add_tail(atom-atom_link, mgr-atoms_list);
@@ -751,7 +753,7 @@ static int atom_begin_and_assign_to_txnh
atom-super = reiser4_get_current_sb();
capture_assign_txnh_nolock(atom, txnh);
- spin_unlock((atom-alock));
+ spin_unlock_atom(atom);
spin_unlock_txnh(txnh);
return -E_REPEAT;
@@ -2112,11 +2114,11 @@ static void fuse_not_fused_lock_owners(t
atomic_inc(atomf-refcount);
spin_unlock_txnh(ctx-trans);
if (atomf atomh) {
- spin_lock_atom(atomf);
+ spin_lock_atom_nested(atomf);
} else {
spin_unlock_atom(atomh);
spin_lock_atom(atomf);
- spin_lock_atom(atomh);
+ spin_lock_atom_nested(atomh);
}
if (atomh == atomf || !atom_isopen(atomh) ||
!atom_isopen(atomf)) {
release_two_atoms(atomf, atomh);
@@ -2794,10 +2796,10 @@ static void lock_two_atoms(txn_atom * on
/* lock the atom with lesser address first */
if (one two) {
spin_lock_atom(one);
- spin_lock_atom(two);
+ spin_lock_atom_nested(two);
} else {
spin_lock_atom(two);
- spin_lock_atom(one);
+ spin_lock_atom_nested(one);
}
}
--- linux-2.6-git.orig/fs/reiser4/txnmgr.h
+++ linux-2.6-git/fs/reiser4/txnmgr.h
@@ -503,6 +503,7 @@ static inline void spin_lock_atom(txn_at
{
/* check that spinlocks of lower priorities are not held */
assert(, (LOCK_CNT_NIL(spin_locked_txnh)
+ LOCK_CNT_NIL(spin_locked_atom)
LOCK_CNT_NIL(spin_locked_jnode)
LOCK_CNT_NIL(spin_locked_zlock)
LOCK_CNT_NIL(rw_locked_dk)
@@ -514,6 +515,20 @@ static inline void spin_lock_atom(txn_at
LOCK_CNT_INC(spin_locked);
}
+static inline void spin_lock_atom_nested(txn_atom *atom)
+{
+ assert(, (LOCK_CNT_NIL(spin_locked_txnh)
+ LOCK_CNT_NIL(spin_locked_jnode)
+ LOCK_CNT_NIL(spin_locked_zlock)
+
Re: Reiser4 stress test.
On Tuesday 22 August 2006 01:23, Hans Reiser wrote: Thanks Andrew, please be patient and persistent with us at this time, as one programmer is on vacation, and the other is only able to work a few hours a day due to an illness. No problem. I'll post what I find to the list; the posts will still be there when you have the time to devote to solving bugs. The delay will do me no harm whatsoever and I may even get to the bottom of one or two bugs in the meantime. (I happen to have time to spare at the moment). Andrew Wade
Re: Reiser4 stress test.
On Tuesday 22 August 2006 01:23, Hans Reiser wrote: Thanks Andrew, please be patient and persistent with us at this time, as one programmer is on vacation, and the other is only able to work a few hours a day due to an illness. No problem. I'll post what I find to the list; the posts will still be there when you have time to devote to chasing bugs. They're not urgent problems for me; I just happen to have the time and interest to devote myself to solving them right now, and it appears I'll be able to muddle through the code okay. Andrew
Re: Reiser4 stress test.
Thanks Andrew, please be patient and persistent with us at this time, as one programmer is on vacation, and the other is only able to work a few hours a day due to an illness. Hans
Re: reiser4 nfsroot problem
Am Donnerstag 17 August 2006 18:35 schrieb Prakash Punnoor: Hi, Using the reiser4 patch 3 for 2.6.17 with a 2.6.18-rc4 kernel (I ported the reiser4 patch by hand with help of mm kernel...) I wasn't able to boot a nfsroot client with the image residing on a master with reiser4 partition. The same setup works w/o problems once I moved the image to an ext3 partition. Perhaps I should note, that the master is x86_64 and the client x86. Do you need more infos to reproduce the problem? -- (°= =°) //\ Prakash Punnoor /\\ V_/ \_V pgpbp4OQBTeHP.pgp Description: PGP signature
Re: reiser4 nfsroot problem
Hello, On 17 August 2006 20:38, Prakash Punnoor wrote: Am Donnerstag 17 August 2006 18:35 schrieb Prakash Punnoor: Hi, Using the reiser4 patch 3 for 2.6.17 with a 2.6.18-rc4 kernel (I ported the reiser4 patch by hand with help of mm kernel...) I wasn't able to boot a nfsroot client with the image residing on a master with reiser4 partition. The same setup works w/o problems once I moved the image to an ext3 partition. Perhaps I should note, that the master is x86_64 and the client x86. Do you need more infos to reproduce the problem? did it work with reiser4 with another kernels, before 2.6.17 ? does plain nfs mount work, not nfsroot? -- Alex.
Re: reiser4 nfsroot problem
Am Donnerstag 17 August 2006 19:36 schrieb Alexander Zarochentsev: Hello, On 17 August 2006 20:38, Prakash Punnoor wrote: Am Donnerstag 17 August 2006 18:35 schrieb Prakash Punnoor: Hi, Using the reiser4 patch 3 for 2.6.17 with a 2.6.18-rc4 kernel (I ported the reiser4 patch by hand with help of mm kernel...) I wasn't able to boot a nfsroot client with the image residing on a master with reiser4 partition. The same setup works w/o problems once I moved the image to an ext3 partition. Perhaps I should note, that the master is x86_64 and the client x86. Do you need more infos to reproduce the problem? did it work with reiser4 with another kernels, before 2.6.17 ? does plain nfs mount work, not nfsroot? I haven't tested it with another kernel. Mounting the export on the master itself and on another maschine in userspace works though. Just nfsroot borks. -- (°= =°) //\ Prakash Punnoor /\\ V_/ \_V pgp5yRfRH72mF.pgp Description: PGP signature
Re: reiser4-2.6.18-rc2-mm1: possible circular locking dependency detected in txn_end
Hello
On Saturday 12 August 2006 17:26, Laurent Riffard wrote:
Le 03.08.2006 17:07, Laurent Riffard a écrit :
Le 03.08.2006 08:09, Alexander Zarochentsev a écrit :
On Tuesday 01 August 2006 01:29, Laurent Riffard wrote:
Le 31.07.2006 21:55, Vladimir V. Saveliev a écrit :
Hello
What kind of load did you run on reiser4 at that time?
I just formatted a new 2GB Reiser4 FS, then I moved a whole ccache
cache tree to this new FS (cache size was about 20~30 Mbytes).
Something like:
# mkfs.reiser4 /dev/vglinux1/ccache
# mount -tauto -onoatime /dev/vglinux1/ccache /mnt/disk
# mv ~laurent/.ccache/* /mnt/disk/
I was not able to reproduce it. Can you please try the following patch?
lock validator friendly locking of new atom in
atom_begin_and_assign_to_txnh and locking of two atoms.
Signed-off-by: Alexander Zarochentsev [EMAIL PROTECTED]
---
fs/reiser4/txnmgr.c | 14 --
fs/reiser4/txnmgr.h | 15 +++
2 files changed, 23 insertions(+), 6 deletions(-)
[patch snipped]
I tried this patch: it's slow as hell (CPU is ~100% system) and it
panics when syncing...
reiser4 panicked cowardly: reiser4[shutdown(1904)]: spin_lock_atom
(fs/reiser4/txmgr.h:509)[]:
Hello,
I tried again with linux 2.6.18-rc3-mm2+hotfixes.
# booted to runlevel 1
~$ mount
...
/dev/mapper/vglinux1-lvhome on /home type reiserfs (rw)
/dev/mapper/vglinux1-lvccache on /home/laurent/.ccache type reiser4
(rw,nosuid,nodev,noatime) ...
~$ df ~/.ccache
FilesystemSize Used Avail Use% Mounted on
/dev/mapper/vglinux1-lvccache
2.0G 53M 1.9G 3% /home/laurent/.ccache
~$ time mv ~/.ccache/* ~/tmp/ccache
0.10user 6.01system 0:07.92elapsed 77%CPU (0avgtext+0avgdata
0maxresident)k 0inputs+0outputs (1major+296minor)pagefaults 0swaps
dmesg output:
===
[ INFO: possible circular locking dependency detected ]
---
mv/1255 is trying to acquire lock:
(txnh-hlock){--..}, at: [e101f0cf] txn_end+0x191/0x368 [reiser4]
but task is already holding lock:
(atom-alock){--..}, at: [e101b674] txnh_get_atom+0xf6/0x39e
[reiser4]
which lock already depends on the new lock.
the existing dependency chain (in reverse order) is:
- #1 (atom-alock){--..}:
[c012ce82] lock_acquire+0x60/0x80
[c0291c08] _spin_lock+0x19/0x28
[e101cc0b] try_capture+0x7cf/0x1cd7 [reiser4]
[e10096e5] longterm_lock_znode+0x427/0x84f [reiser4]
[e1038fe7] seal_validate+0x221/0x5ee [reiser4]
[e10652a1] find_entry+0x126/0x307 [reiser4]
[e1065974] rem_entry_common+0xe9/0x4ba [reiser4]
[e104c9bc] unlink_common+0x258/0x364 [reiser4]
[c015f7bc] vfs_unlink+0x47/0x87
[c01611b4] do_unlinkat+0x8c/0x122
[c016125a] sys_unlink+0x10/0x12
[c0102c39] sysenter_past_esp+0x56/0x8d
- #0 (txnh-hlock){--..}:
[c012ce82] lock_acquire+0x60/0x80
[c0291c08] _spin_lock+0x19/0x28
[e101f0cf] txn_end+0x191/0x368 [reiser4]
[e10109b5] reiser4_exit_context+0x1c2/0x571 [reiser4]
[e104cabd] unlink_common+0x359/0x364 [reiser4]
[c015f7bc] vfs_unlink+0x47/0x87
[c01611b4] do_unlinkat+0x8c/0x122
[c016125a] sys_unlink+0x10/0x12
[c0102c39] sysenter_past_esp+0x56/0x8d
other info that might help us debug this:
3 locks held by mv/1255:
#0: (inode-i_mutex/1){--..}, at: [c0161181]
do_unlinkat+0x59/0x122 #1: (inode-i_mutex){--..}, at: [c0290a94]
mutex_lock+0x21/0x24 #2: (atom-alock){--..}, at: [e101b674]
txnh_get_atom+0xf6/0x39e [reiser4]
stack backtrace:
[c0103a97] show_trace+0xd/0x10
[c0103ff6] dump_stack+0x19/0x1b
[c012c20d] print_circular_bug_tail+0x59/0x64
[c012ca2c] __lock_acquire+0x814/0x9a5
[c012ce82] lock_acquire+0x60/0x80
[c0291c08] _spin_lock+0x19/0x28
[e101f0cf] txn_end+0x191/0x368 [reiser4]
[e10109b5] reiser4_exit_context+0x1c2/0x571 [reiser4]
[e104cabd] unlink_common+0x359/0x364 [reiser4]
[c015f7bc] vfs_unlink+0x47/0x87
[c01611b4] do_unlinkat+0x8c/0x122
[c016125a] sys_unlink+0x10/0x12
[c0102c39] sysenter_past_esp+0x56/0x8d
~$ time sync
0.00user 0.02system 0:00.49elapsed 4%CPU (0avgtext+0avgdata
0maxresident)k 0inputs+0outputs (1major+202minor)pagefaults 0swaps
Move the files backward...
~$ time mv ~/tmp/ccache/* ~/.ccache/
0.11user 3.98system 0:04.09elapsed 100%CPU (0avgtext+0avgdata
0maxresident)k 0inputs+0outputs (0major+286minor)pagefaults 0swaps
~$ time sync
0.00user 0.00system 0:01.86elapsed 0%CPU (0avgtext+0avgdata
0maxresident)k 0inputs+0outputs (0major+204minor)pagefaults 0swaps
So this problem still
Re: reiser4-2.6.18-rc2-mm1: possible circular locking dependency detected in txn_end
Le 14.08.2006 13:50, Vladimir V. Saveliev a écrit :
Hello
On Saturday 12 August 2006 17:26, Laurent Riffard wrote:
Le 03.08.2006 17:07, Laurent Riffard a écrit :
Le 03.08.2006 08:09, Alexander Zarochentsev a écrit :
On Tuesday 01 August 2006 01:29, Laurent Riffard wrote:
Le 31.07.2006 21:55, Vladimir V. Saveliev a écrit :
Hello
What kind of load did you run on reiser4 at that time?
I just formatted a new 2GB Reiser4 FS, then I moved a whole ccache
cache tree to this new FS (cache size was about 20~30 Mbytes).
Something like:
# mkfs.reiser4 /dev/vglinux1/ccache
# mount -tauto -onoatime /dev/vglinux1/ccache /mnt/disk
# mv ~laurent/.ccache/* /mnt/disk/
I was not able to reproduce it. Can you please try the following patch?
lock validator friendly locking of new atom in
atom_begin_and_assign_to_txnh and locking of two atoms.
Signed-off-by: Alexander Zarochentsev [EMAIL PROTECTED]
---
fs/reiser4/txnmgr.c | 14 --
fs/reiser4/txnmgr.h | 15 +++
2 files changed, 23 insertions(+), 6 deletions(-)
[patch snipped]
I tried this patch: it's slow as hell (CPU is ~100% system) and it
panics when syncing...
reiser4 panicked cowardly: reiser4[shutdown(1904)]: spin_lock_atom
(fs/reiser4/txmgr.h:509)[]:
Hello,
I tried again with linux 2.6.18-rc3-mm2+hotfixes.
# booted to runlevel 1
~$ mount
...
/dev/mapper/vglinux1-lvhome on /home type reiserfs (rw)
/dev/mapper/vglinux1-lvccache on /home/laurent/.ccache type reiser4
(rw,nosuid,nodev,noatime) ...
~$ df ~/.ccache
FilesystemSize Used Avail Use% Mounted on
/dev/mapper/vglinux1-lvccache
2.0G 53M 1.9G 3% /home/laurent/.ccache
~$ time mv ~/.ccache/* ~/tmp/ccache
0.10user 6.01system 0:07.92elapsed 77%CPU (0avgtext+0avgdata
0maxresident)k 0inputs+0outputs (1major+296minor)pagefaults 0swaps
dmesg output:
===
[ INFO: possible circular locking dependency detected ]
---
mv/1255 is trying to acquire lock:
(txnh-hlock){--..}, at: [e101f0cf] txn_end+0x191/0x368 [reiser4]
but task is already holding lock:
(atom-alock){--..}, at: [e101b674] txnh_get_atom+0xf6/0x39e
[reiser4]
which lock already depends on the new lock.
the existing dependency chain (in reverse order) is:
- #1 (atom-alock){--..}:
[c012ce82] lock_acquire+0x60/0x80
[c0291c08] _spin_lock+0x19/0x28
[e101cc0b] try_capture+0x7cf/0x1cd7 [reiser4]
[e10096e5] longterm_lock_znode+0x427/0x84f [reiser4]
[e1038fe7] seal_validate+0x221/0x5ee [reiser4]
[e10652a1] find_entry+0x126/0x307 [reiser4]
[e1065974] rem_entry_common+0xe9/0x4ba [reiser4]
[e104c9bc] unlink_common+0x258/0x364 [reiser4]
[c015f7bc] vfs_unlink+0x47/0x87
[c01611b4] do_unlinkat+0x8c/0x122
[c016125a] sys_unlink+0x10/0x12
[c0102c39] sysenter_past_esp+0x56/0x8d
- #0 (txnh-hlock){--..}:
[c012ce82] lock_acquire+0x60/0x80
[c0291c08] _spin_lock+0x19/0x28
[e101f0cf] txn_end+0x191/0x368 [reiser4]
[e10109b5] reiser4_exit_context+0x1c2/0x571 [reiser4]
[e104cabd] unlink_common+0x359/0x364 [reiser4]
[c015f7bc] vfs_unlink+0x47/0x87
[c01611b4] do_unlinkat+0x8c/0x122
[c016125a] sys_unlink+0x10/0x12
[c0102c39] sysenter_past_esp+0x56/0x8d
other info that might help us debug this:
3 locks held by mv/1255:
#0: (inode-i_mutex/1){--..}, at: [c0161181]
do_unlinkat+0x59/0x122 #1: (inode-i_mutex){--..}, at: [c0290a94]
mutex_lock+0x21/0x24 #2: (atom-alock){--..}, at: [e101b674]
txnh_get_atom+0xf6/0x39e [reiser4]
stack backtrace:
[c0103a97] show_trace+0xd/0x10
[c0103ff6] dump_stack+0x19/0x1b
[c012c20d] print_circular_bug_tail+0x59/0x64
[c012ca2c] __lock_acquire+0x814/0x9a5
[c012ce82] lock_acquire+0x60/0x80
[c0291c08] _spin_lock+0x19/0x28
[e101f0cf] txn_end+0x191/0x368 [reiser4]
[e10109b5] reiser4_exit_context+0x1c2/0x571 [reiser4]
[e104cabd] unlink_common+0x359/0x364 [reiser4]
[c015f7bc] vfs_unlink+0x47/0x87
[c01611b4] do_unlinkat+0x8c/0x122
[c016125a] sys_unlink+0x10/0x12
[c0102c39] sysenter_past_esp+0x56/0x8d
~$ time sync
0.00user 0.02system 0:00.49elapsed 4%CPU (0avgtext+0avgdata
0maxresident)k 0inputs+0outputs (1major+202minor)pagefaults 0swaps
Move the files backward...
~$ time mv ~/tmp/ccache/* ~/.ccache/
0.11user 3.98system 0:04.09elapsed 100%CPU (0avgtext+0avgdata
0maxresident)k 0inputs+0outputs (0major+286minor)pagefaults 0swaps
~$ time sync
0.00user 0.00system 0:01.86elapsed 0%CPU (0avgtext+0avgdata
0maxresident)k 0inputs+0outputs (0major+204minor)pagefaults 0swaps
So
Re: reiser4-2.6.18-rc2-mm1: possible circular locking dependency detected in txn_end
Le 03.08.2006 17:07, Laurent Riffard a écrit :
Le 03.08.2006 08:09, Alexander Zarochentsev a écrit :
On Tuesday 01 August 2006 01:29, Laurent Riffard wrote:
Le 31.07.2006 21:55, Vladimir V. Saveliev a écrit :
Hello
What kind of load did you run on reiser4 at that time?
I just formatted a new 2GB Reiser4 FS, then I moved a whole ccache
cache tree to this new FS (cache size was about 20~30 Mbytes).
Something like:
# mkfs.reiser4 /dev/vglinux1/ccache
# mount -tauto -onoatime /dev/vglinux1/ccache /mnt/disk
# mv ~laurent/.ccache/* /mnt/disk/
I was not able to reproduce it. Can you please try the following patch?
lock validator friendly locking of new atom in
atom_begin_and_assign_to_txnh and locking of two atoms.
Signed-off-by: Alexander Zarochentsev [EMAIL PROTECTED]
---
fs/reiser4/txnmgr.c | 14 --
fs/reiser4/txnmgr.h | 15 +++
2 files changed, 23 insertions(+), 6 deletions(-)
[patch snipped]
I tried this patch: it's slow as hell (CPU is ~100% system) and it
panics when syncing...
reiser4 panicked cowardly: reiser4[shutdown(1904)]: spin_lock_atom
(fs/reiser4/txmgr.h:509)[]:
Hello,
I tried again with linux 2.6.18-rc3-mm2+hotfixes.
# booted to runlevel 1
~$ mount
...
/dev/mapper/vglinux1-lvhome on /home type reiserfs (rw)
/dev/mapper/vglinux1-lvccache on /home/laurent/.ccache type reiser4
(rw,nosuid,nodev,noatime)
...
~$ df ~/.ccache
FilesystemSize Used Avail Use% Mounted on
/dev/mapper/vglinux1-lvccache
2.0G 53M 1.9G 3% /home/laurent/.ccache
~$ time mv ~/.ccache/* ~/tmp/ccache
0.10user 6.01system 0:07.92elapsed 77%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (1major+296minor)pagefaults 0swaps
dmesg output:
===
[ INFO: possible circular locking dependency detected ]
---
mv/1255 is trying to acquire lock:
(txnh-hlock){--..}, at: [e101f0cf] txn_end+0x191/0x368 [reiser4]
but task is already holding lock:
(atom-alock){--..}, at: [e101b674] txnh_get_atom+0xf6/0x39e [reiser4]
which lock already depends on the new lock.
the existing dependency chain (in reverse order) is:
- #1 (atom-alock){--..}:
[c012ce82] lock_acquire+0x60/0x80
[c0291c08] _spin_lock+0x19/0x28
[e101cc0b] try_capture+0x7cf/0x1cd7 [reiser4]
[e10096e5] longterm_lock_znode+0x427/0x84f [reiser4]
[e1038fe7] seal_validate+0x221/0x5ee [reiser4]
[e10652a1] find_entry+0x126/0x307 [reiser4]
[e1065974] rem_entry_common+0xe9/0x4ba [reiser4]
[e104c9bc] unlink_common+0x258/0x364 [reiser4]
[c015f7bc] vfs_unlink+0x47/0x87
[c01611b4] do_unlinkat+0x8c/0x122
[c016125a] sys_unlink+0x10/0x12
[c0102c39] sysenter_past_esp+0x56/0x8d
- #0 (txnh-hlock){--..}:
[c012ce82] lock_acquire+0x60/0x80
[c0291c08] _spin_lock+0x19/0x28
[e101f0cf] txn_end+0x191/0x368 [reiser4]
[e10109b5] reiser4_exit_context+0x1c2/0x571 [reiser4]
[e104cabd] unlink_common+0x359/0x364 [reiser4]
[c015f7bc] vfs_unlink+0x47/0x87
[c01611b4] do_unlinkat+0x8c/0x122
[c016125a] sys_unlink+0x10/0x12
[c0102c39] sysenter_past_esp+0x56/0x8d
other info that might help us debug this:
3 locks held by mv/1255:
#0: (inode-i_mutex/1){--..}, at: [c0161181] do_unlinkat+0x59/0x122
#1: (inode-i_mutex){--..}, at: [c0290a94] mutex_lock+0x21/0x24
#2: (atom-alock){--..}, at: [e101b674] txnh_get_atom+0xf6/0x39e
[reiser4]
stack backtrace:
[c0103a97] show_trace+0xd/0x10
[c0103ff6] dump_stack+0x19/0x1b
[c012c20d] print_circular_bug_tail+0x59/0x64
[c012ca2c] __lock_acquire+0x814/0x9a5
[c012ce82] lock_acquire+0x60/0x80
[c0291c08] _spin_lock+0x19/0x28
[e101f0cf] txn_end+0x191/0x368 [reiser4]
[e10109b5] reiser4_exit_context+0x1c2/0x571 [reiser4]
[e104cabd] unlink_common+0x359/0x364 [reiser4]
[c015f7bc] vfs_unlink+0x47/0x87
[c01611b4] do_unlinkat+0x8c/0x122
[c016125a] sys_unlink+0x10/0x12
[c0102c39] sysenter_past_esp+0x56/0x8d
~$ time sync
0.00user 0.02system 0:00.49elapsed 4%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (1major+202minor)pagefaults 0swaps
Move the files backward...
~$ time mv ~/tmp/ccache/* ~/.ccache/
0.11user 3.98system 0:04.09elapsed 100%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+286minor)pagefaults 0swaps
~$ time sync
0.00user 0.00system 0:01.86elapsed 0%CPU (0avgtext+0avgdata 0maxresident)k
0inputs+0outputs (0major+204minor)pagefaults 0swaps
So this problem still appears in 2.6.18-rc3-mm2+hotfixes.
I applied the patch zam sent 01 August 2006. Compile, boot to
runlevel 1 and test again. The warning
Re: Ebuild/rpm/deb repo's (was Re: reiser4 can now bear with filled fs, looks stable to me...)
On Wed, Aug 09, 2006 at 12:01:42AM -0400, David Masover wrote: A warning isn't good? Would you rather it be an error? Of course not. It merely appears inconsistent to offer a root fs choice that may cause severe problems at bootup time. When I went to install my first SuSE (brand-new 6.1 at that time) I looked up the huge printed manual: what are my options? OK there weren't many at the time ; ) But there was the 1023cyl problem, being a careful guy I opted for a separate /boot at the start of the disk, for example. Better safe than sorry. And if there had been a choice in terms of file systems (ext2 was the only way afair) there would have been a decision as well. Now say I carefully weigh different aspects of different solutions, only to find out that my decision was not so good and will possibly cause problems _unless_ I play around with strange mount options that _disable_ some of the features supporting my decision. Knowing trouble lies ahead of course is better than just running into it : ) Some people would like to test Grub's XFS support... Sure thing. But those might not want to do this on their production machine, anyway, and test something more bleeding-edge than what distros ship at the time. Presuming there is ongoing development, that is. It's not necessarily broken, just potentially unreliable, and difficult to work with (you have to set arcane mount options or somesuch). Same for ReiserFS3, by the way. All that, especially unreliable, is not something your average user is likely to accept. People have been cursing the whole x86-pc hardware for the mess that mainstream software (M$ mainly, but no need to copy their mistakes) has been for the past two decades. I've been asked by folks more than once if switching to a mac will ease their pain. So what a distro installer wants to do is remove all the options from anything but expert mode that might cause the slightest hickup. Really, while Grub is useful, it's a rather large duplication-of-code effort. XOSL is even moreso, especially considering it doesn't support Linux or multiboot natively -- it must boot Grub or Lilo in order to run Linux or HURD. Why aren't we using kexec for this already? This is way beyond my knowledge, sorry. But I like kexec a lot, just wonder how to get to the point where kexec can be used _without_ linux up and running already... It's called backup and restore. Or did you have a different idea for how to convert an existing installation to another root FS, or do a fresh installation without nuking your partitions anyway? Well, there is convertfs, fsconvert (which i.e. _is_ backuprestore), parted, etc. There are ways, some are safe, most are not. But I am afraid we have gone way OT by now. What I had in mind all the way basically was the distro user / end user experience which should be as uncomplicated as possible, while still leaving choices. But when it comes to advanced file systems, these choices come at the price of a more sophisticated setup or lack of reliability, as you already pointed out. But how do you explain to a linux newbie: use ext2|3 for / due to bootloader issues _or_ something funky, yet with funkyness partly turned off for bootloader compatibility, _or_ use a tiny ext2 /boot and feel free to fsckmount as you damn please. Most people just don't care anyway, and those who do will want to know why. And telling them that otherwise it won't work (without extra care) yields the question, why not?, and you might _not_ want to try explaining the entire situation from within the installer help text. Or maybe that would be the way to go? Who knows how much people actually read in the end, before they just quit - worrying, deciding, or even installing the thing at all. And now, go easy on me! This was never intended as a debate of colliding opinions - call it brainstorming : ) Kind regards, Chris pgpsSTvFCVQnV.pgp Description: PGP signature
Re: Ebuild/rpm/deb repo's (was Re: reiser4 can now bear with filled fs, looks stable to me...)
On Mon, 2006-08-07 at 20:05 -0400, David Masover wrote: Maciej Sołtysiak wrote: Hello David, Tuesday, August 8, 2006, 1:23:01 AM, you wrote: Sounds good. I don't have an ubuntu to test with at the moment, though. Well, both MS Virtual PC and VMWare are free of charge, so installing is a real snap. Under what, though? I don't want MS crap on my OS X (need that for work ATM), and I can't imagine they've ported it to Linux. I have no reason to boot Windows except for games, and if I was going to do that, I may as well shrink my Windows partition to make room for a native install. Which would be fine, but it's a lot of work when I don't run Ubuntu normally. I'd be willing to test on the one Ubuntu server I run, but it's across the country until next week, and also work-critical. Not to nitpick, but isn't that emulation? Or have they actually done real virtualization yet? I don't know the differences, can you shed some light? AFAICS M$ will be shipping Virtual PC with Vista to allow people run older software under virtual machines. (be it virtualized or emulated) Still hard to say. Virtualization splits up the real hardware. It's like a scheduler, only for OSes. Emulation is more like an interpreter -- it reads each instruction and then executes something that does the same thing. Emulation can work from any arch to any arch, so Rosetta (allowing PPC OS X apps to run on OS X86) is emulation. Emulation is usually at least 2x slower than native. Virtualization usually approaches native for CPU stuff, but at least disk IO and graphics usually have to be emulated -- so no 3D acceleration, so no games under a guest OS. If MS wanted to do the best possible thing for their consumers, they'd give you a free XP under VirtualPC with Vista, and actually do virtualization. If M$ wanted to make it even more likely for people to want to upgrade to Vista, they might deliberately make it cost tons of money and make it emulation, so that XP looks slower, and native Vista apps look so much faster that people complain until everything works on Vista. If Virtual PC is emulation, maybe Virtual Server 2005 R2 (also free of charge) is virtualizaton. I have no idea what Virtual Server is. There are many forms of virtualization see 1, one of them being emulation. VMware and virtual pc do emulation but if possible they will pass instruction directly to the hardware without emulating vias a hypervisor. Good reading on virtualization on 2. Greets Sander [1] http://en.wikipedia.org/wiki/Virtualization [2] http://www.kernelthread.com/publications/virtualization/
Re: Ebuild/rpm/deb repo's (was Re: reiser4 can now bear with filled fs, looks stable to me...)
On Mon, Aug 07, 2006 at 01:09:42PM -0400, David Masover wrote: Christian Trefzer wrote: Few people keep a 32MB ext2 for /boot purposes these days, so it really is imperative that grub can read kernel images off a reiser4 /. I think there are patches, but I do keep a 32 meg ext3 for /boot, because it seems like no matter what FS I choose, there's some sort of caveat involving Grub. I know when installing XFS as a root FS on Ubuntu, it talks about Grub problems... New installations is what I had in mind there. People see they could use something fancy, but their bootloader-du-jour won't take it. Bummer. The warning message is the only thing keeping folks from running into a broken installation. No good. I mean, having Grub support everything would be nice, but if you're reformatting anyway, I don't think it's that imperative. Yeah, _if_ you are s.o. who knows how to turn a partition table upside down without losing a single bit of data. Even then, it is quite a hassle to move the start of a partition 4 cylinders up to make room for an all-compatible ext2 /boot ; ) Grub is a bootloader and as such should (as an optimum) be able to grab kernels off of any fs. I guess patches are accepted by upstream developers? Kind regards, Chris pgpcCiJHSyAXR.pgp Description: PGP signature
Re: Ebuild/rpm/deb repo's (was Re: reiser4 can now bear with filled fs, looks stable to me...)
On 8/8/06, Christian Trefzer [EMAIL PROTECTED] wrote: Grub is a bootloader and as such should (as an optimum) be able to grab kernels off of any fs. I guess patches are accepted by upstream developers? Grub v1 (The one we all know) is alpha status according to their devs. Its no longer under active development - in favour of v2. AFAIK, they have stopped accepting patches for it, and they have no plans to release v2 in the near future. and I don't think it is even compilable That doesn't stop mainline distros to accept namesys' patch anyway. Sarath
Re: Ebuild/rpm/deb repo's (was Re: reiser4 can now bear with filled fs, looks stable to me...)
Hello David, Tuesday, August 8, 2006, 2:05:23 AM, you wrote: Under what, though? I don't want MS crap on my OS X (need that for work ATM), and I can't imagine they've ported it to Linux. I have no reason to boot Windows except for games, and if I was going to do that, I may as well shrink my Windows partition to make room for a native install. VMware has announced a product designed to enable Mac OS X users to run multiple PC operating systems while Microsoft is putting a halt to a version of its Virtual PC software for Intel-based Macs. http://www.osnews.com/story.php?news_id=15417 It seems that: 1) ms has virtual pc for macs. 2) vmware will have. -- Best regards, Maciej
Re: Ebuild/rpm/deb repo's (was Re: reiser4 can now bear with filled fs, looks stable to me...)
On Sun, Aug 06, 2006 at 04:23:16PM +0200, Maciej Sołtysiak wrote: I tried to create a kernel package with reiser4 for ubuntu-server (dapper) They ship a 2.6.15 (heavily modified) kernel upon which the current reiser4-for-2.6.17-3.patch applies fine but unfortunately miscompiles, eg. fs/reiser4/plugin/file_ops_readdir.c: In function 'llseek_common_dir': fs/reiser4/plugin/file_ops_readdir.c:486: warning: implicit declaration of function 'mutex_lock' fs/reiser4/plugin/file_ops_readdir.c:486: error: 'struct inode' has no member named 'i_mutex' fs/reiser4/plugin/file_ops_readdir.c:508: warning: implicit declaration of function 'mutex_unlock' fs/reiser4/plugin/file_ops_readdir.c:508: error: 'struct inode' has no member named 'i_mutex' I bet these are trivial to fix and I will try to do this but my time constraints currently prevent me from doing that. Guess these won't be so easy, afaik 2.6.15 lacks the semaphore-mutex transition. There also is an issue with grub. The kernel alone is fine for creating partitions (or loop devices) but with grub not patched we can't install boot partitions. No biggy, I guess, but still a problem. Few people keep a 32MB ext2 for /boot purposes these days, so it really is imperative that grub can read kernel images off a reiser4 /. I could create a vanilla + reiser4 kernel package easily but that would be stripped off of the important dapper/debian patches and that is a no-no for dapper users, I guess. At least for non-guru freaks who run their own, modified kernels. Sure. The best thing might be to back-port internal fixes in fs/reiser4/ to reiser4-for-2.6.15-x so the API calls match the kernel. Patching 2.6.15 with a 2.6.17 patch is a no-no. But you _are_ taking a step forward, and your effort is appreciated : D Kind regards, Chris pgpaKpM0N5nTR.pgp Description: PGP signature
Re: Ebuild/rpm/deb repo's (was Re: reiser4 can now bear with filled fs, looks stable to me...)
Christian Trefzer wrote: On Sun, Aug 06, 2006 at 04:23:16PM +0200, Maciej Sołtysiak wrote: There also is an issue with grub. The kernel alone is fine for creating partitions (or loop devices) but with grub not patched we can't install boot partitions. No biggy, I guess, but still a problem. Few people keep a 32MB ext2 for /boot purposes these days, so it really is imperative that grub can read kernel images off a reiser4 /. I think there are patches, but I do keep a 32 meg ext3 for /boot, because it seems like no matter what FS I choose, there's some sort of caveat involving Grub. I know when installing XFS as a root FS on Ubuntu, it talks about Grub problems... I mean, having Grub support everything would be nice, but if you're reformatting anyway, I don't think it's that imperative.
Re: Ebuild/rpm/deb repo's (was Re: reiser4 can now bear with filled fs, looks stable to me...)
Hello David, Monday, August 7, 2006, 7:09:42 PM, you wrote: I mean, having Grub support everything would be nice, but if you're reformatting anyway, I don't think it's that imperative. I have come up to that conclusion too. If someone would be getting an r4-enabled kernel on an already installed system they would not care much for grub support unless they have only one root partition. I have built today an r4-patched ubuntu kernel package (yes, debs!) using edgy eft's git-based build system, which allows me to build ubuntu kernels! Yes, with all the stuff they have in there, with all the configuration versions they support (-386, -k7, -server, etc...) I booted one and I will try to create an r4 partition later today. Please note, that this is done all under virtualization (Microsoft Virtual PC). -- Best regards, Maciej
Re: Ebuild/rpm/deb repo's (was Re: reiser4 can now bear with filled fs, looks stable to me...)
Maciej Sołtysiak wrote: Hello David, hi I have built today an r4-patched ubuntu kernel package (yes, debs!) Sounds good. I don't have an ubuntu to test with at the moment, though. Please note, that this is done all under virtualization (Microsoft Virtual PC). Not to nitpick, but isn't that emulation? Or have they actually done real virtualization yet?
Re: Ebuild/rpm/deb repo's (was Re: reiser4 can now bear with filled fs, looks stable to me...)
Hello David, Tuesday, August 8, 2006, 1:23:01 AM, you wrote: Sounds good. I don't have an ubuntu to test with at the moment, though. Well, both MS Virtual PC and VMWare are free of charge, so installing is a real snap. Not to nitpick, but isn't that emulation? Or have they actually done real virtualization yet? I don't know the differences, can you shed some light? AFAICS M$ will be shipping Virtual PC with Vista to allow people run older software under virtual machines. (be it virtualized or emulated) If Virtual PC is emulation, maybe Virtual Server 2005 R2 (also free of charge) is virtualizaton. -- Best regards, Maciej
Re: Ebuild/rpm/deb repo's (was Re: reiser4 can now bear with filled fs, looks stable to me...)
Maciej Sołtysiak wrote: Hello David, Tuesday, August 8, 2006, 1:23:01 AM, you wrote: Sounds good. I don't have an ubuntu to test with at the moment, though. Well, both MS Virtual PC and VMWare are free of charge, so installing is a real snap. Under what, though? I don't want MS crap on my OS X (need that for work ATM), and I can't imagine they've ported it to Linux. I have no reason to boot Windows except for games, and if I was going to do that, I may as well shrink my Windows partition to make room for a native install. Which would be fine, but it's a lot of work when I don't run Ubuntu normally. I'd be willing to test on the one Ubuntu server I run, but it's across the country until next week, and also work-critical. Not to nitpick, but isn't that emulation? Or have they actually done real virtualization yet? I don't know the differences, can you shed some light? AFAICS M$ will be shipping Virtual PC with Vista to allow people run older software under virtual machines. (be it virtualized or emulated) Still hard to say. Virtualization splits up the real hardware. It's like a scheduler, only for OSes. Emulation is more like an interpreter -- it reads each instruction and then executes something that does the same thing. Emulation can work from any arch to any arch, so Rosetta (allowing PPC OS X apps to run on OS X86) is emulation. Emulation is usually at least 2x slower than native. Virtualization usually approaches native for CPU stuff, but at least disk IO and graphics usually have to be emulated -- so no 3D acceleration, so no games under a guest OS. If MS wanted to do the best possible thing for their consumers, they'd give you a free XP under VirtualPC with Vista, and actually do virtualization. If M$ wanted to make it even more likely for people to want to upgrade to Vista, they might deliberately make it cost tons of money and make it emulation, so that XP looks slower, and native Vista apps look so much faster that people complain until everything works on Vista. If Virtual PC is emulation, maybe Virtual Server 2005 R2 (also free of charge) is virtualizaton. I have no idea what Virtual Server is.
Re: Ebuild/rpm/deb repo's (was Re: reiser4 can now bear with filled fs, looks stable to me...)
I know it is very easy to create ubuntu kernel packages (I have done a few) I might try to do one for current dapper kernel for i386. But it would have to wait due to time my personal constraints (projects, etc.) Answering myself... I tried to create a kernel package with reiser4 for ubuntu-server (dapper) They ship a 2.6.15 (heavily modified) kernel upon which the current reiser4-for-2.6.17-3.patch applies fine but unfortunately miscompiles, eg. fs/reiser4/plugin/file_ops_readdir.c: In function 'llseek_common_dir': fs/reiser4/plugin/file_ops_readdir.c:486: warning: implicit declaration of function 'mutex_lock' fs/reiser4/plugin/file_ops_readdir.c:486: error: 'struct inode' has no member named 'i_mutex' fs/reiser4/plugin/file_ops_readdir.c:508: warning: implicit declaration of function 'mutex_unlock' fs/reiser4/plugin/file_ops_readdir.c:508: error: 'struct inode' has no member named 'i_mutex' I bet these are trivial to fix and I will try to do this but my time constraints currently prevent me from doing that. There also is an issue with grub. The kernel alone is fine for creating partitions (or loop devices) but with grub not patched we can't install boot partitions. No biggy, I guess, but still a problem. I could create a vanilla + reiser4 kernel package easily but that would be stripped off of the important dapper/debian patches and that is a no-no for dapper users, I guess. At least for non-guru freaks who run their own, modified kernels. -- Best regards, Maciej
Re: reiser4: maybe just fix bugs?
Theodore Tso wrote: On Tue, Aug 01, 2006 at 11:55:57AM -0500, David Masover wrote: If I understand it right, the original Reiser4 model of file metadata is the file-as-directory stuff that caused such a furor the last big push for inclusion (search for Silent semantic changes in Reiser4): The furor was caused by concerns Al Viro expressed about locking/deadlock issues that reiser4 introduced. Which, I believe, was about file-as-dir. Which also had problems with things like directory loops. That's sort of a disk space memory leak. The bigger issue with xattr support is two-fold. First of all, there are the progams that are expecting the existing extended attribute interface, Yeah... More importantly are the system-level extended attributes, such as those used by SELINUX, which by definition are not supposed to be visible to the user at all, I don't see why either of these are issues. The SELINUX stuff can be a plugin which doesn't necessarily have a user-level interface. Cryptocompress, for instance, exists independent of its user-level interface (probably the file-as-dir stuff), and will probably be implemented in some sort of stable form as a system-wide default for new files. So, certainly metadata (xattrs) as a plugin could be implemented with no UI at all, or any given UI. ... Anyway, I still see no reason why these cannot be implemented in Reiser4, other than the possibility that if it uses plugins, I guarantee that at least one or two people will hate the implementation for that reason alone. Not supporting xattrs means that those distro's that use SELINUX by default (i.e., RHEL, Fedora, etc.) won't want to use reiser4, because SELINUX won't work on reiser4 filesytstems. Right. So they will be implemented, eventually. Whether or not Hans cares about this is up to him He does, or he should. Reiser4 needs every bit of acceptance it can get right now, as long as it can get them without compromising its goals or philosophy. Extended attributes only compromise these because it provides less incentive to learn any other metadata interface that Reiser4 provides. But that's irrelevant if Reiser4 doesn't gain enough acceptance due to lack of xattr support, anything it has will be irrelevant anyway. So just as we provide the standard interface to Unix permissions (even though we intend to implement things like acls and views, and even though there was a file/.pseudo/rwx interface), we should provide the standard xattr interface, and the standard direct IO interface, and anything else that's practical. Be a good, standard filesystem first, and an innovative filesystem second.
Re: reiser4-2.6.18-rc2-mm1: possible circular locking dependency detected in txn_end
On Tuesday 01 August 2006 01:29, Laurent Riffard wrote:
Le 31.07.2006 21:55, Vladimir V. Saveliev a écrit :
Hello
What kind of load did you run on reiser4 at that time?
I just formatted a new 2GB Reiser4 FS, then I moved a whole ccache
cache tree to this new FS (cache size was about 20~30 Mbytes).
Something like:
# mkfs.reiser4 /dev/vglinux1/ccache
# mount -tauto -onoatime /dev/vglinux1/ccache /mnt/disk
# mv ~laurent/.ccache/* /mnt/disk/
I was not able to reproduce it. Can you please try the following patch?
lock validator friendly locking of new atom in
atom_begin_and_assign_to_txnh and locking of two atoms.
Signed-off-by: Alexander Zarochentsev [EMAIL PROTECTED]
---
fs/reiser4/txnmgr.c | 14 --
fs/reiser4/txnmgr.h | 15 +++
2 files changed, 23 insertions(+), 6 deletions(-)
Index: linux-2.6-git/fs/reiser4/txnmgr.c
===
--- linux-2.6-git.orig/fs/reiser4/txnmgr.c
+++ linux-2.6-git/fs/reiser4/txnmgr.c
@@ -730,10 +730,12 @@ static int atom_begin_and_assign_to_txnh
assert(jmacd-17, atom_isclean(atom));
/*
-* do not use spin_lock_atom because we have broken lock ordering here
-* which is ok, as long as @atom is new and inaccessible for others.
+* lock ordering is broken here. It is ok, as long as @atom is new
+* and inaccessible for others. We can't use spin_lock_atom or
+* spin_lock(atom-alock) because they care about locking
+* dependencies. spin_trylock_lock doesn't.
*/
- spin_lock((atom-alock));
+ check_me(, spin_trylock_atom(atom));
/* add atom to the end of transaction manager's list of atoms */
list_add_tail(atom-atom_link, mgr-atoms_list);
@@ -749,7 +751,7 @@ static int atom_begin_and_assign_to_txnh
atom-super = reiser4_get_current_sb();
capture_assign_txnh_nolock(atom, txnh);
- spin_unlock((atom-alock));
+ spin_unlock_atom(atom);
spin_unlock_txnh(txnh);
return -E_REPEAT;
@@ -2791,10 +2793,10 @@ static void lock_two_atoms(txn_atom * on
/* lock the atom with lesser address first */
if (one two) {
spin_lock_atom(one);
- spin_lock_atom(two);
+ spin_lock_atom_nested(two);
} else {
spin_lock_atom(two);
- spin_lock_atom(one);
+ spin_lock_atom_nested(one);
}
}
Index: linux-2.6-git/fs/reiser4/txnmgr.h
===
--- linux-2.6-git.orig/fs/reiser4/txnmgr.h
+++ linux-2.6-git/fs/reiser4/txnmgr.h
@@ -502,6 +502,7 @@ static inline void spin_lock_atom(txn_at
{
/* check that spinlocks of lower priorities are not held */
assert(, (LOCK_CNT_NIL(spin_locked_txnh)
+ LOCK_CNT_NIL(spin_locked_atom)
LOCK_CNT_NIL(spin_locked_jnode)
LOCK_CNT_NIL(spin_locked_zlock)
LOCK_CNT_NIL(rw_locked_dk)
@@ -513,6 +514,20 @@ static inline void spin_lock_atom(txn_at
LOCK_CNT_INC(spin_locked);
}
+static inline void spin_lock_atom_nested(txn_atom *atom)
+{
+ assert(, (LOCK_CNT_NIL(spin_locked_txnh)
+ LOCK_CNT_NIL(spin_locked_jnode)
+ LOCK_CNT_NIL(spin_locked_zlock)
+ LOCK_CNT_NIL(rw_locked_dk)
+ LOCK_CNT_NIL(rw_locked_tree)));
+
+ spin_lock_nested((atom-alock), SINGLE_DEPTH_NESTING);
+
+ LOCK_CNT_INC(spin_locked_atom);
+ LOCK_CNT_INC(spin_locked);
+}
+
static inline int spin_trylock_atom(txn_atom *atom)
{
if (spin_trylock((atom-alock))) {
Re: reiser4: maybe just fix bugs?
On Tue, Aug 01, 2006 at 11:55:57AM -0500, David Masover wrote: If I understand it right, the original Reiser4 model of file metadata is the file-as-directory stuff that caused such a furor the last big push for inclusion (search for Silent semantic changes in Reiser4): The furor was caused by concerns Al Viro expressed about locking/deadlock issues that reiser4 introduced. The bigger issue with xattr support is two-fold. First of all, there are the progams that are expecting the existing extended attribute interface, and not implementing it will simply mean that as far as Samba, and other applications, Reiser4 simply will not haved xattr support. More importantly are the system-level extended attributes, such as those used by SELINUX, which by definition are not supposed to be visible to the user at all, but only are supposed to be there for the SELINUX (or some other kernel layer, in general) to access. Not supporting xattrs means that those distro's that use SELINUX by default (i.e., RHEL, Fedora, etc.) won't want to use reiser4, because SELINUX won't work on reiser4 filesytstems. Whether or not Hans cares about this is up to him - Ted
Re: Ebuild/rpm/deb repo's (was Re: reiser4 can now bear with filled fs, looks stable to me...)
Am Dienstag, 1. August 2006 23:59 schrieb Sander Sweers: On Tue, 2006-08-01 at 23:12 +0200, Maciej Sołtysiak wrote: [...] Are there any on the list who know of rpm's for Suse/Redhat/Mandrake that include reiser4? Suse excluded reiser4 from 10.1 because they want to keep the kernel cleaner than in previous versions. Il ask what's the status with 10.2, but I think it's still left out. With the new build-server (http://en.opensuse.org/Build_Service) it shouldn't be too hard to build packages. Also Suse supports now special packages for kernel-modules, which can even survive a kernel-update, if the API does not change. But I see another important point: Reiser4 is very fast if you have more than one reiser4 partition. But: most home users make one or two reiser4 partitions for testing purpose. Until the whole system does not run on reiser4 the speed improvement cannot be felt 100%. So community members should make pressure at Suse, Fedora, Mandriva to get reiser4 supported by the distro. -- Üdvözlettel -- Mit freundlichen Grüssen, Marcel Hilzinger
Re: Ebuild/rpm/deb repo's (was Re: reiser4 can now bear with filled fs, looks stable to me...)
Am Donnerstag, 3. August 2006 10:55 schrieb Marcel Hilzinger: Am Dienstag, 1. August 2006 23:59 schrieb Sander Sweers: On Tue, 2006-08-01 at 23:12 +0200, Maciej Sołtysiak wrote: [...] Are there any on the list who know of rpm's for Suse/Redhat/Mandrake that include reiser4? One more idea: The next release of Ubuntu is a playground release. Hans, perhaps you should have a meeting with Mark Shuttleworth. Reiser4 inclusion in Linspire was a big step, but Linspire has not really a community. If you get Reiser4 included in the next Ubuntu release (and can make it rock stable then!), you do not have to bother about Fedora or Suse... It's quite late for inclusion in the next Ubuntu release, but who knows. -- Üdvözlettel -- Mit freundlichen Grüssen, Marcel Hilzinger
Re: Ebuild/rpm/deb repo's (was Re: reiser4 can now bear with filled fs, looks stable to me...)
Marcel Hilzinger wrote: Am Donnerstag, 3. August 2006 10:55 schrieb Marcel Hilzinger: Am Dienstag, 1. August 2006 23:59 schrieb Sander Sweers: On Tue, 2006-08-01 at 23:12 +0200, Maciej Sołtysiak wrote: [...] Are there any on the list who know of rpm's for Suse/Redhat/Mandrake that include reiser4? One more idea: The next release of Ubuntu is a playground release. Hans, perhaps you should have a meeting with Mark Shuttleworth. Reiser4 inclusion in Linspire was a big step, but Linspire has not really a community. If you get Reiser4 included in the next Ubuntu release (and can make it rock stable then!), you do not have to bother about Fedora or Suse... It's quite late for inclusion in the next Ubuntu release, but who knows. Could you contact him for us, and ask? It is more convincing when users ask. Hans
Re: Ebuild/rpm/deb repo's (was Re: reiser4 can now bear with filled fs, looks stable to me...)
It's quite late for inclusion in the next Ubuntu release, but who knows. Maybe it is not, it's a playground, Mark would not hesitate to postpone Edgy's release if it requires polishing the whole thing due to edgy features. Could you contact him for us, and ask? It is more convincing when users ask. Hehe, Hans, I tried contacting him. On his website he points to his secratary Claire. She's on maternity leave, so another girl, also named Claire replies. Here's what I got for asking about Mark collaborating with Hans: Hi Maciej Thank you for your e-mail and interest in Ubuntu. Unfortunately, this is not something we can help you with. I hope that you are able to find the help you require elsewhere. kind regards Claire God damn it, secrataries... I haven't found Mark's email address anywhere, did anyone have? -- mks
Re: Ebuild/rpm/deb repo's (was Re: reiser4 can now bear with filled fs, looks stable to me...)
On 03/08/06, Maciej Sołtysiak [EMAIL PROTECTED] wrote: It's quite late for inclusion in the next Ubuntu release, but who knows. Maybe it is not, it's a playground, Mark would not hesitate to postpone Edgy's release if it requires polishing the whole thing due to edgy features. Could you contact him for us, and ask? It is more convincing when users ask. Hehe, Hans, I tried contacting him. On his website he points to his secratary Claire. She's on maternity leave, so another girl, also named Claire replies. Here's what I got for asking about Mark collaborating with Hans: Hi Maciej Thank you for your e-mail and interest in Ubuntu. Unfortunately, this is not something we can help you with. I hope that you are able to find the help you require elsewhere. kind regards Claire God damn it, secrataries... I haven't found Mark's email address anywhere, did anyone have? Maybe it is better to get in contact with the 2 people from this link. https://wiki.ubuntu.com/Reiser4 David is a former gentoo users and was the one who maintained the love-sources. The only e-mail address I could find for him was [EMAIL PROTECTED] (his jabber but it looks ok. Greets Sander
Re: reiser4-2.6.18-rc2-mm1: possible circular locking dependency detected in txn_end
Le 03.08.2006 08:09, Alexander Zarochentsev a écrit :
On Tuesday 01 August 2006 01:29, Laurent Riffard wrote:
Le 31.07.2006 21:55, Vladimir V. Saveliev a écrit :
Hello
What kind of load did you run on reiser4 at that time?
I just formatted a new 2GB Reiser4 FS, then I moved a whole ccache
cache tree to this new FS (cache size was about 20~30 Mbytes).
Something like:
# mkfs.reiser4 /dev/vglinux1/ccache
# mount -tauto -onoatime /dev/vglinux1/ccache /mnt/disk
# mv ~laurent/.ccache/* /mnt/disk/
I was not able to reproduce it. Can you please try the following patch?
lock validator friendly locking of new atom in
atom_begin_and_assign_to_txnh and locking of two atoms.
Signed-off-by: Alexander Zarochentsev [EMAIL PROTECTED]
---
fs/reiser4/txnmgr.c | 14 --
fs/reiser4/txnmgr.h | 15 +++
2 files changed, 23 insertions(+), 6 deletions(-)
Index: linux-2.6-git/fs/reiser4/txnmgr.c
===
--- linux-2.6-git.orig/fs/reiser4/txnmgr.c
+++ linux-2.6-git/fs/reiser4/txnmgr.c
@@ -730,10 +730,12 @@ static int atom_begin_and_assign_to_txnh
assert(jmacd-17, atom_isclean(atom));
/*
- * do not use spin_lock_atom because we have broken lock ordering here
- * which is ok, as long as @atom is new and inaccessible for others.
+ * lock ordering is broken here. It is ok, as long as @atom is new
+ * and inaccessible for others. We can't use spin_lock_atom or
+ * spin_lock(atom-alock) because they care about locking
+ * dependencies. spin_trylock_lock doesn't.
*/
- spin_lock((atom-alock));
+ check_me(, spin_trylock_atom(atom));
/* add atom to the end of transaction manager's list of atoms */
list_add_tail(atom-atom_link, mgr-atoms_list);
@@ -749,7 +751,7 @@ static int atom_begin_and_assign_to_txnh
atom-super = reiser4_get_current_sb();
capture_assign_txnh_nolock(atom, txnh);
- spin_unlock((atom-alock));
+ spin_unlock_atom(atom);
spin_unlock_txnh(txnh);
return -E_REPEAT;
@@ -2791,10 +2793,10 @@ static void lock_two_atoms(txn_atom * on
/* lock the atom with lesser address first */
if (one two) {
spin_lock_atom(one);
- spin_lock_atom(two);
+ spin_lock_atom_nested(two);
} else {
spin_lock_atom(two);
- spin_lock_atom(one);
+ spin_lock_atom_nested(one);
}
}
Index: linux-2.6-git/fs/reiser4/txnmgr.h
===
--- linux-2.6-git.orig/fs/reiser4/txnmgr.h
+++ linux-2.6-git/fs/reiser4/txnmgr.h
@@ -502,6 +502,7 @@ static inline void spin_lock_atom(txn_at
{
/* check that spinlocks of lower priorities are not held */
assert(, (LOCK_CNT_NIL(spin_locked_txnh)
+ LOCK_CNT_NIL(spin_locked_atom)
LOCK_CNT_NIL(spin_locked_jnode)
LOCK_CNT_NIL(spin_locked_zlock)
LOCK_CNT_NIL(rw_locked_dk)
@@ -513,6 +514,20 @@ static inline void spin_lock_atom(txn_at
LOCK_CNT_INC(spin_locked);
}
+static inline void spin_lock_atom_nested(txn_atom *atom)
+{
+ assert(, (LOCK_CNT_NIL(spin_locked_txnh)
+ LOCK_CNT_NIL(spin_locked_jnode)
+ LOCK_CNT_NIL(spin_locked_zlock)
+ LOCK_CNT_NIL(rw_locked_dk)
+ LOCK_CNT_NIL(rw_locked_tree)));
+
+ spin_lock_nested((atom-alock), SINGLE_DEPTH_NESTING);
+
+ LOCK_CNT_INC(spin_locked_atom);
+ LOCK_CNT_INC(spin_locked);
+}
+
static inline int spin_trylock_atom(txn_atom *atom)
{
if (spin_trylock((atom-alock))) {
I tried this patch: it's slow as hell (CPU is ~100% system) and it
panics when syncing...
reiser4 panicked cowardly: reiser4[shutdown(1904)]: spin_lock_atom
(fs/reiser4/txmgr.h:509)[]:
--
laurent
Re: Ebuild/rpm/deb repo's (was Re: reiser4 can now bear with filled fs, looks stable to me...)
Sander Sweers wrote: With the approval of Namesys I would like to add a new entry to the wiki frontpage. It would be very appreciated.
Re: reiser4: maybe just fix bugs?
Andrew Morton wrote: On Mon, 31 Jul 2006 10:26:55 +0100 Denis Vlasenko [EMAIL PROTECTED] wrote: The reiser4 thread seem to be longer than usual. Meanwhile here's poor old me trying to find another four hours to finish reviewing the thing. Thanks Andrew. The writeout code is ugly, although that's largely due to a mismatch between what reiser4 wants to do and what the VFS/MM expects it to do. I agree --- both with it being ugly, and that being part of why. If it works, we can live with it, although perhaps the VFS could be made smarter. I would be curious regarding any ideas on that. Next time I read through that code, I will keep in mind that you are open to making VFS changes if it improves things, and I will try to get clever somehow and send it by you. Our squalloc code though is I must say the most complicated and ugliest piece of code I ever worked on for which every cumulative ugliness had a substantive performance advantage requiring us to keep it. If you spare yourself from reading that, it is understandable to do so. I'd say that resier4's major problem is the lack of xattrs, acls and direct-io. That's likely to significantly limit its vendor uptake. (As might the copyright assignment thing, but is that a kernel.org concern?) Thanks to you and the batch write code, direct io support will now be much easier to code, and it probably will get coded the soonest of those features. acls are on the todo list, but doing them right might require solving a few additional issues (finishing the inheritance code, etc.) The plugins appear to be wildly misnamed - they're just an internal abstraction layer which permits later feature additions to be added in a clean and safe manner. Certainly not worth all this fuss. Could I suggest that further technical critiques of reiser4 include a file-and-line reference? That should ease the load on vger. Thanks.
Re: reiser4 can now bear with filled fs, looks stable to me...
Hello David, Monday, July 31, 2006, 11:46:34 PM, you wrote: You must be new here... ;-) I wanted to point out that because: Options B and C are all that ever seems to happen when reiserfs-list and lkml collide. and: The speed of a nonworking program is irrelevant. The cost-effectiveness of an impossible solution is irrelevant. maybe the more important thing is to allow people use r4 on their own (rpms, debs, apt/gentoo/repositories, etc.) better, than to push that hard for kernel inclusion. Currently the r4 patch is very easy to apply, you can apply it on top of heavily patched kernels with no or little fuzz, which is very good. But, as Hans wrote earlier, not every user knows how to patch, but in ubuntu for example it is fairly easy (and encouraged by the official forums/wikis) for those users to add additional repositories using synaptic or adept or editing /etc/apt/sources.list. I mean, there were huge objections against FUSE too, remember? But Miklos built a steady and growing userbase. Maybe that is something to realize, Hans, we don't need kernel inclusion to have a growing userbase. (or atleast a steady one) A side note: the only time we had not reiserfs-list and lkml collide (that much) was when Andrew Morton was commenting and when Cristoph made a list of things to phix - that cost people some nerve but it nevertheles was more productive than the usual flamewars. -- Best regards, Maciej
Re: reiser4: maybe just fix bugs?
Hello On Mon, 2006-07-31 at 20:18 -0600, Hans Reiser wrote: Andrew Morton wrote: The writeout code is ugly, although that's largely due to a mismatch between what reiser4 wants to do and what the VFS/MM expects it to do. Yes. reiser4 writeouts atoms. Most of pages get into atoms via sys_write. But pages dirtied via shared mapping do not. They get into atoms in reiser4's writepages address space operation. That is why reiser4_sync_inodes has two steps: on first one it calls generic_sync_sb_inodes to call writepages for dirty inodes to capture pages dirtied via shared mapping into atoms. Second step flushes atoms. I agree --- both with it being ugly, and that being part of why. If it works, we can live with it, although perhaps the VFS could be made smarter. I would be curious regarding any ideas on that. Next time I read through that code, I will keep in mind that you are open to making VFS changes if it improves things, and I will try to get clever somehow and send it by you. Our squalloc code though is I must say the most complicated and ugliest piece of code I ever worked on for which every cumulative ugliness had a substantive performance advantage requiring us to keep it. If you spare yourself from reading that, it is understandable to do so. I'd say that resier4's major problem is the lack of xattrs, acls and direct-io. That's likely to significantly limit its vendor uptake. xattrs is really a problem.
Re: reiser4: maybe just fix bugs?
On Tue, 01 Aug 2006 15:24:37 +0400 Vladimir V. Saveliev [EMAIL PROTECTED] wrote: The writeout code is ugly, although that's largely due to a mismatch between what reiser4 wants to do and what the VFS/MM expects it to do. Yes. reiser4 writeouts atoms. Most of pages get into atoms via sys_write. But pages dirtied via shared mapping do not. They get into atoms in reiser4's writepages address space operation. It think you mean -writepage - reiser4 desn't implement -writepages(). I assume you considered hooking into -set_page_dirty() to do the add-to-atom thing earlier on? We'll merge mm-tracking-shared-dirty-pages.patch into 2.6.19-rc1, which would make that approach considerably more successful, I expect. -set_page_dirty() is a bit awkward because it can be called under spinlock. Maybe comething could also be gained from the new vm_operations_struct.page_mkwrite(), although that's less obvious... That is why reiser4_sync_inodes has two steps: on first one it calls generic_sync_sb_inodes to call writepages for dirty inodes to capture pages dirtied via shared mapping into atoms. Second step flushes atoms. I agree --- both with it being ugly, and that being part of why. If it works, we can live with it, although perhaps the VFS could be made smarter. I would be curious regarding any ideas on that. Next time I read through that code, I will keep in mind that you are open to making VFS changes if it improves things, and I will try to get clever somehow and send it by you. Our squalloc code though is I must say the most complicated and ugliest piece of code I ever worked on for which every cumulative ugliness had a substantive performance advantage requiring us to keep it. If you spare yourself from reading that, it is understandable to do so. I'd say that resier4's major problem is the lack of xattrs, acls and direct-io. That's likely to significantly limit its vendor uptake. xattrs is really a problem. That's not good. The ability to properly support SELinux is likely to be important.
Re: reiser4: maybe just fix bugs?
Hello On Tue, 2006-08-01 at 07:33 -0700, Andrew Morton wrote: On Tue, 01 Aug 2006 15:24:37 +0400 Vladimir V. Saveliev [EMAIL PROTECTED] wrote: The writeout code is ugly, although that's largely due to a mismatch between what reiser4 wants to do and what the VFS/MM expects it to do. Yes. reiser4 writeouts atoms. Most of pages get into atoms via sys_write. But pages dirtied via shared mapping do not. They get into atoms in reiser4's writepages address space operation. It think you mean -writepage - reiser4 desn't implement -writepages(). no. there is one. It is reiser4/plugin/file/file.c:writepages_unix_file(). reiser4_writepage just kicks kernel thread (entd) which works similar to reiser4_sync_inodes() (described earlier) and waits until several pages (including the one reiser4_writepage is called with) are written. I assume you considered hooking into -set_page_dirty() to do the add-to-atom thing earlier on? Currently, add-to-atom is not simple. It may require memory allocations and disk i/o-s. I guess these are not supposed to be called in -set_page_dirty(). That is why in reiser4_set_page_dirty we just mark the page in mapping's tree and dealy adding to atoms until flush time. We'll merge mm-tracking-shared-dirty-pages.patch into 2.6.19-rc1, which would make that approach considerably more successful, I expect. -set_page_dirty() is a bit awkward because it can be called under spinlock. Maybe comething could also be gained from the new vm_operations_struct.page_mkwrite(), although that's less obvious... That is why reiser4_sync_inodes has two steps: on first one it calls generic_sync_sb_inodes to call writepages for dirty inodes to capture pages dirtied via shared mapping into atoms. Second step flushes atoms. I agree --- both with it being ugly, and that being part of why. If it works, we can live with it, although perhaps the VFS could be made smarter. I would be curious regarding any ideas on that. Next time I read through that code, I will keep in mind that you are open to making VFS changes if it improves things, and I will try to get clever somehow and send it by you. Our squalloc code though is I must say the most complicated and ugliest piece of code I ever worked on for which every cumulative ugliness had a substantive performance advantage requiring us to keep it. If you spare yourself from reading that, it is understandable to do so. I'd say that resier4's major problem is the lack of xattrs, acls and direct-io. That's likely to significantly limit its vendor uptake. xattrs is really a problem. That's not good. The ability to properly support SELinux is likely to be important. Do you think that if reiser4 supported xattrs - it would increase its chances on inclusion? PS: what exactly did you refer to when you said that writeout code is ugly?
Re: reiser4: maybe just fix bugs?
Vladimir V. Saveliev wrote: Do you think that if reiser4 supported xattrs - it would increase its chances on inclusion? Probably the opposite. If I understand it right, the original Reiser4 model of file metadata is the file-as-directory stuff that caused such a furor the last big push for inclusion (search for Silent semantic changes in Reiser4): foo.mp3/.../rwx# permissions foo.mp3/.../artist # part of the id3 tag So I suspect xattrs would just be a different interface to this stuff, maybe just a subset of it (to prevent namespace collisions): foo.mp3/.../xattr/ # contains files representing attributes Of course, you'd be able to use the standard interface for getting/setting these. The point is, I don't think Hans/Namesys wants to do this unless they're going to do it right, especially because they already have the file-as-dir stuff somewhat done. Note that these are neither mutually exclusive nor mutually dependent -- you don't have to enable file-as-dir to make xattrs work. I know it's not done yet, though. I can understand Hans dragging his feet here, because xattrs and traditional acls are examples of things Reiser4 is supposed to eventually replace. Anyway, if xattrs were done now, the only good that would come of it is building a userbase outside the vanilla kernel. I can't see it as doing anything but hurting inclusion by introducing more confusion about plugins. I could be entirely wrong, though. I speak for neither Hans/Namesys/reiserfs nor LKML. Talk amongst yourselves...
Ebuild/rpm/deb repo's (was Re: reiser4 can now bear with filled fs, looks stable to me...)
On Tue, 2006-08-01 at 13:28 +0200, Maciej Sołtysiak wrote: Hello David, Monday, July 31, 2006, 11:46:34 PM, you wrote: You must be new here... ;-) I wanted to point out that because: Options B and C are all that ever seems to happen when reiserfs-list and lkml collide. and: The speed of a nonworking program is irrelevant. The cost-effectiveness of an impossible solution is irrelevant. maybe the more important thing is to allow people use r4 on their own (rpms, debs, apt/gentoo/repositories, etc.) better, than to push that hard for kernel inclusion. Yes, and in case of gentoo there are already people maintaining an ebuild which pull in r4 on the wiki. http://gentoo-wiki.com/HOWTO_Reiser4_With_Gentoo-Sources WHen you make it easy for people to use reiser4 by providing ebuilds, rpm's or deb's more users will be tempted to try out reiser4 who would normally not be able or willing to patch the kernel. Maintaning an ebuild for example is easy. And adding in another patch to a kernel deb/rpm should also not be too difficult. It will take some time to do each month but sacrificing a few hours to update these to me would be worth it. Maybe the reiser cummunity can help out the namesys devs? Greets Sander
Re: reiser4: maybe just fix bugs?
On 8/1/06, Andrew Morton [EMAIL PROTECTED] wrote: On Tue, 01 Aug 2006 15:24:37 +0400 Vladimir V. Saveliev [EMAIL PROTECTED] wrote: The writeout code is ugly, although that's largely due to a mismatch between what reiser4 wants to do and what the VFS/MM expects it to do. Yes. reiser4 writeouts atoms. Most of pages get into atoms via sys_write. But pages dirtied via shared mapping do not. They get into atoms in reiser4's writepages address space operation. It think you mean -writepage - reiser4 desn't implement -writepages(). I assume you considered hooking into -set_page_dirty() to do the add-to-atom thing earlier on? We'll merge mm-tracking-shared-dirty-pages.patch into 2.6.19-rc1, which would make that approach considerably more successful, I expect. -set_page_dirty() is a bit awkward because it can be called under spinlock. Maybe comething could also be gained from the new vm_operations_struct.page_mkwrite(), although that's less obvious... That is why reiser4_sync_inodes has two steps: on first one it calls generic_sync_sb_inodes to call writepages for dirty inodes to capture pages dirtied via shared mapping into atoms. Second step flushes atoms. I agree --- both with it being ugly, and that being part of why. If it works, we can live with it, although perhaps the VFS could be made smarter. I would be curious regarding any ideas on that. Next time I read through that code, I will keep in mind that you are open to making VFS changes if it improves things, and I will try to get clever somehow and send it by you. Our squalloc code though is I must say the most complicated and ugliest piece of code I ever worked on for which every cumulative ugliness had a substantive performance advantage requiring us to keep it. If you spare yourself from reading that, it is understandable to do so. I'd say that resier4's major problem is the lack of xattrs, acls and direct-io. That's likely to significantly limit its vendor uptake. xattrs is really a problem. That's not good. The ability to properly support SELinux is likely to be important. i disagreee that it will be difficult. unfortunately, the patch that I am working on right now, which fixes the various reiser4 specific functions to avoid using VFS data structures unless needed, is a prerequisite to enabling xattrs. creating it is a time of tedium for me, and it will cause a bit of internal churn (1000 lines and counting). it's all in the fs/reiser4 directory though, and it should cause minimal disruption, as far as runtime bugs introduced. once that's taken care of, i will be delighted to enable xattr support in a way that will make selinux and beagle and such run as expected, and will have the added advantage of some major scalability improvements for certain lookup and update operations. NATE
Re: reiser4-2.6.18-rc2-mm1: possible circular locking dependency detected in txn_end
Hello Ingo,
there is a new reiser4 / lock validator problem:
On Sunday 30 July 2006 22:57, Laurent Riffard wrote:
===
[ INFO: possible circular locking dependency detected ]
---
mv/29012 is trying to acquire lock:
(txnh-hlock){--..}, at: [e0c8e09b] txn_end+0x191/0x368 [reiser4]
but task is already holding lock:
(atom-alock){--..}, at: [e0c8a640] txnh_get_atom+0xf6/0x39e
[reiser4]
which lock already depends on the new lock.
it is absolutely legal in reiser4 to lock atom first, then lock
transaction handle.
i guess the lock validator recorded wrong dependency rule from one place
where the spinlocks are taken in reverse order. that place is in
fs/reiser4/txnmgr.c:atom_begin_and_assign_to_txnh, that atom is new,
just kmalloc'ed object which is inaccessible for others, so it can't a
source for deadlock.
but how to explain that to the lock validator?
the existing dependency chain (in reverse order) is:
- #1 (atom-alock){--..}:
[c012ce2f] lock_acquire+0x60/0x80
[c0292968] _spin_lock+0x19/0x28
[e0c8bbd7] try_capture+0x7cf/0x1cd7 [reiser4]
[e0c786e1] longterm_lock_znode+0x427/0x84f [reiser4]
[e0ca55dc] coord_by_handle+0x2be/0x7f7 [reiser4]
[e0ca5f89] coord_by_key+0x1e3/0x22d [reiser4]
[e0c7dbd2] insert_by_key+0x8f/0xe0 [reiser4]
[e0cbf7f1] write_sd_by_inode_common+0x361/0x61a [reiser4]
[e0cbfce4] create_object_common+0xf1/0xf6 [reiser4]
[e0cbaebf] create_vfs_object+0x51d/0x732 [reiser4]
[e0cbb1fd] mkdir_common+0x43/0x4b [reiser4]
[c015ed33] vfs_mkdir+0x5a/0x9d
[c0160f5e] sys_mkdirat+0x88/0xc0
[c0160fa6] sys_mkdir+0x10/0x12
[c0102c2d] sysenter_past_esp+0x56/0x8d
- #0 (txnh-hlock){--..}:
[c012ce2f] lock_acquire+0x60/0x80
[c0292968] _spin_lock+0x19/0x28
[e0c8e09b] txn_end+0x191/0x368 [reiser4]
[e0c7f97d] reiser4_exit_context+0x1c2/0x571 [reiser4]
[e0cbb091] create_vfs_object+0x6ef/0x732 [reiser4]
[e0cbb1fd] mkdir_common+0x43/0x4b [reiser4]
[c015ed33] vfs_mkdir+0x5a/0x9d
[c0160f5e] sys_mkdirat+0x88/0xc0
[c0160fa6] sys_mkdir+0x10/0x12
[c0102c2d] sysenter_past_esp+0x56/0x8d
other info that might help us debug this:
2 locks held by mv/29012:
#0: (inode-i_mutex/1){--..}, at: [c015f50b]
lookup_create+0x1d/0x73
#1: (atom-alock){--..}, at: [e0c8a640]
txnh_get_atom+0xf6/0x39e [reiser4]
stack backtrace:
[c0104df0] show_trace+0xd/0x10
[c0104e0c] dump_stack+0x19/0x1d
[c012bc62] print_circular_bug_tail+0x59/0x64
[c012cc3e] __lock_acquire+0x814/0x9a5
[c012ce2f] lock_acquire+0x60/0x80
[c0292968] _spin_lock+0x19/0x28
[e0c8e09b] txn_end+0x191/0x368 [reiser4]
[e0c7f97d] reiser4_exit_context+0x1c2/0x571 [reiser4]
[e0cbb091] create_vfs_object+0x6ef/0x732 [reiser4]
[e0cbb1fd] mkdir_common+0x43/0x4b [reiser4]
[c015ed33] vfs_mkdir+0x5a/0x9d
[c0160f5e] sys_mkdirat+0x88/0xc0
[c0160fa6] sys_mkdir+0x10/0x12
[c0102c2d] sysenter_past_esp+0x56/0x8d
(Linux antares.localdomain 2.6.18-rc2-mm1 #77 Sun Jul 30 15:09:34
CEST 2006 i686 AMD Athlon(TM) XP 1600+ unknown GNU/Linux)
--
Alex.
Re: reiser4: maybe just fix bugs?
On 8/1/06, David Masover [EMAIL PROTECTED] wrote: Vladimir V. Saveliev wrote: Do you think that if reiser4 supported xattrs - it would increase its chances on inclusion? Probably the opposite. If I understand it right, the original Reiser4 model of file metadata is the file-as-directory stuff that caused such a furor the last big push for inclusion (search for Silent semantic changes in Reiser4): foo.mp3/.../rwx# permissions foo.mp3/.../artist # part of the id3 tag So I suspect xattrs would just be a different interface to this stuff, maybe just a subset of it (to prevent namespace collisions): foo.mp3/.../xattr/ # contains files representing attributes Of course, you'd be able to use the standard interface for getting/setting these. The point is, I don't think Hans/Namesys wants to do this unless they're going to do it right, especially because they already have the file-as-dir stuff somewhat done. Note that these are neither mutually exclusive nor mutually dependent -- you don't have to enable file-as-dir to make xattrs work. I know it's not done yet, though. I can understand Hans dragging his feet here, because xattrs and traditional acls are examples of things Reiser4 is supposed to eventually replace. Anyway, if xattrs were done now, the only good that would come of it is building a userbase outside the vanilla kernel. I can't see it as doing anything but hurting inclusion by introducing more confusion about plugins. I could be entirely wrong, though. I speak for neither Hans/Namesys/reiserfs nor LKML. Talk amongst yourselves... i should clarify things a bit here. yes, hans' goal is for there to be no difference between the xattr namespace and the readdir one. unfortunately, this is not feasible with the current VFS, and some major work would have to be done to enable this without some pathological cases cropping up. some very smart people think that it cannot be done at all. xattr is a seperate VFS interface, which avoids those problems by defining certain restrictions on how the 'files' which live in that namespace can be manupulated. for instance, hard links are non-existent, and the 'mv' command cannot move a file between different xattr namespaces. enabling xattr would have no connection to the file-as-directory stuff, and (without extra work) would not even allow access to the things reiser4 defined in the '...' directory. also enabling xattr in the way i intend would in no way compromise hans' long-term vision. HOWEVER, i *need* to point out that hans and i disagree somewhat on the specifics here, and so i should say adamently i don't speak here on behalf of hans or namesys. that won't stop me from submitting my own patch though :) NATE
Re: Ebuild/rpm/deb repo's (was Re: reiser4 can now bear with filled fs, looks stable to me...)
Hello Sander, Tuesday, August 1, 2006, 8:10:34 PM, you wrote: Yes, and in case of gentoo there are already people maintaining an ebuild which pull in r4 on the wiki. http://gentoo-wiki.com/HOWTO_Reiser4_With_Gentoo-Sources Debian has reiser4progs and kernel-patch-2.6-reiser4: - stable: 20040813-6 - testing: 20050715-1 - unstable: 20050715-1 Very old patches. Also the patch descriptions says: WARNING: this software is to be considered usable but its deployment in production environments is still not recommended. Use at your own risk. I know it is very easy to create ubuntu kernel packages (I have done a few) I might try to do one for current dapper kernel for i386. But it would have to wait due to time my personal constraints (projects, etc.) -- Best regards, Maciej
Re: Ebuild/rpm/deb repo's (was Re: reiser4 can now bear with filled fs, looks stable to me...)
On Tue, 2006-08-01 at 23:12 +0200, Maciej Sołtysiak wrote: Hello Sander, Hey Tuesday, August 1, 2006, 8:10:34 PM, you wrote: Yes, and in case of gentoo there are already people maintaining an ebuild which pull in r4 on the wiki. http://gentoo-wiki.com/HOWTO_Reiser4_With_Gentoo-Sources Debian has reiser4progs and kernel-patch-2.6-reiser4: Nice :) - stable: 20040813-6 - testing: 20050715-1 - unstable: 20050715-1 Ouch :( It is in serious need of updating. With the approval of Namesys I would like to add a new entry to the wiki frontpage. I would be someting like Get reiser4 now or Howto install reiser4. Under that we detail the steps to get kernels for distros which include reiser4 and how to patch it yourself. I know it is very easy to create ubuntu kernel packages (I have done a few) I might try to do one for current dapper kernel for i386. But it would have to wait due to time my personal constraints (projects, etc.) Great :) Are there any on the list who know of rpm's for Suse/Redhat/Mandrake that include reiser4? Greets Sander
Re: reiser4: maybe just fix bugs?
Nate Diller wrote: On 8/1/06, David Masover [EMAIL PROTECTED] wrote: Vladimir V. Saveliev wrote: I could be entirely wrong, though. I speak for neither Hans/Namesys/reiserfs nor LKML. Talk amongst yourselves... i should clarify things a bit here. yes, hans' goal is for there to be no difference between the xattr namespace and the readdir one. unfortunately, this is not feasible with the current VFS, and some major work would have to be done to enable this without some pathological cases cropping up. some very smart people think that it cannot be done at all. But an xattr interface should work just fine, even if the rest of the system is inaccessible (no readdir interface) -- preventing all these pathological problems, except the one where Hans implements it the way I'm thinking, and kernel people hate it.
Re: reiser4 can now bear with filled fs, looks stable to me...
I think that most of our problem is that we are too socially insulated from lkml. They are a herd, and decide things based on what thoughts echo most loudly. That none of the shy developers working for me actively post on lkml hurts us quite a bit. It might even be socially effective to shut down reiserfs-list until inclusion occurs. Hans
Re: reiser4 can now bear with filled fs, looks stable to me...
Hans Reiser wrote: I think that most of our problem is that we are too socially insulated from lkml. They are a herd, and decide things based on what thoughts echo most loudly. To be fair, it's not the whole lkml you have to convince, just the few people directly responsible for filesystems and 2.6 maintenance. But then, they probably do consider what the herd is saying... It might even be socially effective to shut down reiserfs-list until inclusion occurs. Maybe. It will be an inconvenience for me, if we have to. I'm not even on LKML, and I'd rather not be -- even this list can get noisy at times... But I will go with it if it's what works best.
Re: reiser4 can now bear with filled fs, looks stable to me...
On Mon, 2006-07-31 at 03:08 -0500, David Masover wrote: Hans Reiser wrote: It might even be socially effective to shut down reiserfs-list until inclusion occurs. Maybe. It will be an inconvenience for me, if we have to. I'm not even on LKML, and I'd rather not be -- even this list can get noisy at times... But I will go with it if it's what works best. I agree with David, for whatever that is worth (I'm just a user of ReiserFS, not a developer). --Dan
Re: reiser4-2.6.18-rc2-mm1: possible circular locking dependency detected in txn_end
Hello
What kind of load did you run on reiser4 at that time?
On Sun, 2006-07-30 at 20:57 +0200, Laurent Riffard wrote:
===
[ INFO: possible circular locking dependency detected ]
---
mv/29012 is trying to acquire lock:
(txnh-hlock){--..}, at: [e0c8e09b] txn_end+0x191/0x368 [reiser4]
but task is already holding lock:
(atom-alock){--..}, at: [e0c8a640] txnh_get_atom+0xf6/0x39e
[reiser4]
which lock already depends on the new lock.
the existing dependency chain (in reverse order) is:
- #1 (atom-alock){--..}:
[c012ce2f] lock_acquire+0x60/0x80
[c0292968] _spin_lock+0x19/0x28
[e0c8bbd7] try_capture+0x7cf/0x1cd7 [reiser4]
[e0c786e1] longterm_lock_znode+0x427/0x84f [reiser4]
[e0ca55dc] coord_by_handle+0x2be/0x7f7 [reiser4]
[e0ca5f89] coord_by_key+0x1e3/0x22d [reiser4]
[e0c7dbd2] insert_by_key+0x8f/0xe0 [reiser4]
[e0cbf7f1] write_sd_by_inode_common+0x361/0x61a [reiser4]
[e0cbfce4] create_object_common+0xf1/0xf6 [reiser4]
[e0cbaebf] create_vfs_object+0x51d/0x732 [reiser4]
[e0cbb1fd] mkdir_common+0x43/0x4b [reiser4]
[c015ed33] vfs_mkdir+0x5a/0x9d
[c0160f5e] sys_mkdirat+0x88/0xc0
[c0160fa6] sys_mkdir+0x10/0x12
[c0102c2d] sysenter_past_esp+0x56/0x8d
- #0 (txnh-hlock){--..}:
[c012ce2f] lock_acquire+0x60/0x80
[c0292968] _spin_lock+0x19/0x28
[e0c8e09b] txn_end+0x191/0x368 [reiser4]
[e0c7f97d] reiser4_exit_context+0x1c2/0x571 [reiser4]
[e0cbb091] create_vfs_object+0x6ef/0x732 [reiser4]
[e0cbb1fd] mkdir_common+0x43/0x4b [reiser4]
[c015ed33] vfs_mkdir+0x5a/0x9d
[c0160f5e] sys_mkdirat+0x88/0xc0
[c0160fa6] sys_mkdir+0x10/0x12
[c0102c2d] sysenter_past_esp+0x56/0x8d
other info that might help us debug this:
2 locks held by mv/29012:
#0: (inode-i_mutex/1){--..}, at: [c015f50b]
lookup_create+0x1d/0x73
#1: (atom-alock){--..}, at: [e0c8a640]
txnh_get_atom+0xf6/0x39e [reiser4]
stack backtrace:
[c0104df0] show_trace+0xd/0x10
[c0104e0c] dump_stack+0x19/0x1d
[c012bc62] print_circular_bug_tail+0x59/0x64
[c012cc3e] __lock_acquire+0x814/0x9a5
[c012ce2f] lock_acquire+0x60/0x80
[c0292968] _spin_lock+0x19/0x28
[e0c8e09b] txn_end+0x191/0x368 [reiser4]
[e0c7f97d] reiser4_exit_context+0x1c2/0x571 [reiser4]
[e0cbb091] create_vfs_object+0x6ef/0x732 [reiser4]
[e0cbb1fd] mkdir_common+0x43/0x4b [reiser4]
[c015ed33] vfs_mkdir+0x5a/0x9d
[c0160f5e] sys_mkdirat+0x88/0xc0
[c0160fa6] sys_mkdir+0x10/0x12
[c0102c2d] sysenter_past_esp+0x56/0x8d
(Linux antares.localdomain 2.6.18-rc2-mm1 #77 Sun Jul 30 15:09:34
CEST 2006 i686 AMD Athlon(TM) XP 1600+ unknown GNU/Linux)
Re: reiser4-2.6.18-rc2-mm1: possible circular locking dependency detected in txn_end
Le 31.07.2006 21:55, Vladimir V. Saveliev a écrit :
Hello
What kind of load did you run on reiser4 at that time?
I just formatted a new 2GB Reiser4 FS, then I moved a whole ccache
cache tree to this new FS (cache size was about 20~30 Mbytes).
Something like:
# mkfs.reiser4 /dev/vglinux1/ccache
# mount -tauto -onoatime /dev/vglinux1/ccache /mnt/disk
# mv ~laurent/.ccache/* /mnt/disk/
On Sun, 2006-07-30 at 20:57 +0200, Laurent Riffard wrote:
===
[ INFO: possible circular locking dependency detected ]
---
mv/29012 is trying to acquire lock:
(txnh-hlock){--..}, at: [e0c8e09b] txn_end+0x191/0x368 [reiser4]
but task is already holding lock:
(atom-alock){--..}, at: [e0c8a640] txnh_get_atom+0xf6/0x39e
[reiser4]
which lock already depends on the new lock.
the existing dependency chain (in reverse order) is:
- #1 (atom-alock){--..}:
[c012ce2f] lock_acquire+0x60/0x80
[c0292968] _spin_lock+0x19/0x28
[e0c8bbd7] try_capture+0x7cf/0x1cd7 [reiser4]
[e0c786e1] longterm_lock_znode+0x427/0x84f [reiser4]
[e0ca55dc] coord_by_handle+0x2be/0x7f7 [reiser4]
[e0ca5f89] coord_by_key+0x1e3/0x22d [reiser4]
[e0c7dbd2] insert_by_key+0x8f/0xe0 [reiser4]
[e0cbf7f1] write_sd_by_inode_common+0x361/0x61a [reiser4]
[e0cbfce4] create_object_common+0xf1/0xf6 [reiser4]
[e0cbaebf] create_vfs_object+0x51d/0x732 [reiser4]
[e0cbb1fd] mkdir_common+0x43/0x4b [reiser4]
[c015ed33] vfs_mkdir+0x5a/0x9d
[c0160f5e] sys_mkdirat+0x88/0xc0
[c0160fa6] sys_mkdir+0x10/0x12
[c0102c2d] sysenter_past_esp+0x56/0x8d
- #0 (txnh-hlock){--..}:
[c012ce2f] lock_acquire+0x60/0x80
[c0292968] _spin_lock+0x19/0x28
[e0c8e09b] txn_end+0x191/0x368 [reiser4]
[e0c7f97d] reiser4_exit_context+0x1c2/0x571 [reiser4]
[e0cbb091] create_vfs_object+0x6ef/0x732 [reiser4]
[e0cbb1fd] mkdir_common+0x43/0x4b [reiser4]
[c015ed33] vfs_mkdir+0x5a/0x9d
[c0160f5e] sys_mkdirat+0x88/0xc0
[c0160fa6] sys_mkdir+0x10/0x12
[c0102c2d] sysenter_past_esp+0x56/0x8d
other info that might help us debug this:
2 locks held by mv/29012:
#0: (inode-i_mutex/1){--..}, at: [c015f50b]
lookup_create+0x1d/0x73
#1: (atom-alock){--..}, at: [e0c8a640]
txnh_get_atom+0xf6/0x39e [reiser4]
stack backtrace:
[c0104df0] show_trace+0xd/0x10
[c0104e0c] dump_stack+0x19/0x1d
[c012bc62] print_circular_bug_tail+0x59/0x64
[c012cc3e] __lock_acquire+0x814/0x9a5
[c012ce2f] lock_acquire+0x60/0x80
[c0292968] _spin_lock+0x19/0x28
[e0c8e09b] txn_end+0x191/0x368 [reiser4]
[e0c7f97d] reiser4_exit_context+0x1c2/0x571 [reiser4]
[e0cbb091] create_vfs_object+0x6ef/0x732 [reiser4]
[e0cbb1fd] mkdir_common+0x43/0x4b [reiser4]
[c015ed33] vfs_mkdir+0x5a/0x9d
[c0160f5e] sys_mkdirat+0x88/0xc0
[c0160fa6] sys_mkdir+0x10/0x12
[c0102c2d] sysenter_past_esp+0x56/0x8d
(Linux antares.localdomain 2.6.18-rc2-mm1 #77 Sun Jul 30 15:09:34
CEST 2006 i686 AMD Athlon(TM) XP 1600+ unknown GNU/Linux)
--
laurent
Re: reiser4 can now bear with filled fs, looks stable to me...
Hello David, Monday, July 31, 2006, 1:30:47 AM, you wrote: Amen. I do not want to see Reiser4 not succeed because of politics, and it really looks like the only way to win the political war is not to play. The technical stuff is really the last way in, but neither side has said anything technical in awhile. The most technical things that have happened lately is Hans pointing to benchmarks and LKML pointing to ext3 plugins. So let's do the math (cost-wise, time-wise) - it is cheaper to develop: a) quality patches b) distro rpms c) repositories with patched sources and binary kernels for debian/ubuntu d) other - it is more expensive to: a) succeed at kernel inclusion b) argue c) waste time -- Best regards, Maciej
Re: reiser4 can now bear with filled fs, looks stable to me...
Maciej Sołtysiak wrote: Hello David, - it is more expensive to: a) succeed at kernel inclusion b) argue c) waste time You must be new here... Options B and C are all that ever seems to happen when reiserfs-list and lkml collide. Is option A possible? The speed of a nonworking program is irrelevant. The cost-effectiveness of an impossible solution is irrelevant.
Re: reiser4 can now bear with filled fs, looks stable to me...
Christian Trefzer wrote: Hi, I booted 2.6.18-rc2-mm1 today and later filled up my /opt partition by accident, and guess what, reiser4 did not screw up : D Hmm, I'm curious, though... How does it react to a few billion files? Sorry, I can't test this, but I will be testing MythTV, if not now, then in a few weeks. Congratulations and thanks to the namesys developers! Hans, I can somewhat understand how you feel about your situation. Don't let frustration get in your way, your work is simply too great. You're an [...] screwing over society ; ) Sometimes you just have to swallow your pride instead of wasting your time by yelling at the rest of the world, and if humble work does not lead to success, there won't be any other way, I fear. Amen. I do not want to see Reiser4 not succeed because of politics, and it really looks like the only way to win the political war is not to play. The technical stuff is really the last way in, but neither side has said anything technical in awhile. The most technical things that have happened lately is Hans pointing to benchmarks and LKML pointing to ext3 plugins. I suspect part of this is simply the word plugin coming around to bite us in the ass, but whatever. We're all tired of this fight. IMHO it would be best to deliver quality patches against all kinds of sources (distro kernels, vanilla -rcs maybe, etc.) Well, we have the patches against vanilla, which seem to work well with at least a few other patches I've tried. and the entire patched source tarball as well, for people to download and build. Next step would be to provide binary packages, and repos for people to add to their package manager's source list. Until distros pick up their respective patch, this is as far as support can go, I guess. That would actually be pretty good, for anyone making the conscious decision to use a filesystem. Still need official distro support to get the people who don't (think they) care. So, what do you all say? Sounds good. I don't have any idea of the work required, either...
Re: reiser4 can now bear with filled fs, looks stable to me...
Christian Trefzer wrote on 07/30/06 15:38: Hi, I booted 2.6.18-rc2-mm1 today and later filled up my /opt partition by accident, and guess what, reiser4 did not screw up : D Lucky you. I tried -rc2-mm1 today, and out of habit, the first thing I do is an fsck on my reiser4 partitions. And that resulted in a crash :-( I am now back to -rc1-mm1, which was the last one that's stable for me. -Joe
Re: portage tree (Was: Re: reiser4 status (correction))
Thanks Christian. You can go ahead and add something to our wiki pointing to it if you would like. This might help tide people over until the repacker ships. Hans
Re: reiser4 status (correction)
Mike Benoit wrote: On Sat, 2006-07-22 at 07:34 -0500, David Masover wrote: The compression will probably mostly be about speed. Remember, if we're talking about people who want to see tangible, visceral results, we're probably also talking about end-users. And trust me, the vast majority of most of my data (as an end-user) is not very compressible. Sure, mine too. Between the many gigs of MP3s and movies I have stored on my HD, only about 10-20GB is the OS, Email, and documents/source code. Even just compressing that small portion though I could probably save between 5-10GB. The difference is though I can do a df before, and a df after, and I can instantly see I got my moneys worth. Same with encryption. I am looking forward to the first user email complaining that he compressed a file stored on reiser4 and it didn't save space (and then someday maybe even an email saying that he compressed a file and it took up more space but the user space compressor is sure that it saved space and he does not understand).:) With the repacker it is much more difficult (for average users) to time how long a program takes to load some file (or launch), before the repacker and after. I think you are confusing repacking and compression? Repacking, by removing seeks, will make it more predictable not less. Especially since caching comes in to play. Also, according to this little poll on one of the compressed FUSE sites you linked to, more people are looking to compression for space saving, then for speed: http://parallel.vub.ac.be/~johan/compFUSEd/index.php?option=pollstask=resultspollid=31 No, mostly we're talking about things like office documents, the majority of which fit in less than a gigabyte, and multimedia (music, movies, games) which will gain very little from compression. If anything, the benefit would be mostly in compressing software. less tangible like fragmentation percentages and minor I/O throughput improvements. I used to work at a large, world wide web hosting company and I could see making a case to management for purchasing Reiser4 compression would be pretty easy for our shared servers. Instantly freeing up large amounts of disk space (where .html/.php files were the vast majority) would save huge amounts of money on disk drives, especially since most of the servers used RAID1 and adding new drives was a huge pain in the neck. Making a case to purchase a repacker would be much, much more difficult. Hmm, the problem is, if storage space is really the big deal, it's been done before, and some of these efforts are still usable and free: http://parallel.vub.ac.be/~johan/compFUSEd/ http://www.miio.net/fusecompress/ http://north.one.pl/~kazik/pub/LZOlayer/ http://apfs.humorgraficojr.com/apfs_ingles.html And while we're on the topic, here's an FS that does unpacking of archives, probably about the same way we imagined it in Reiser4 pseudofiles/magic: http://www.nongnu.org/unpackfs/ But regardless, as far as I can tell, the only real, tangible benefit of using Reiser4 compression instead of one of those four FUSE filesystems is speed. Reiser4 would compress/decompress when actually hitting the disk, not just the FS, and it would also probably use in-kernel compression, rather than calling out to userspace on every FS operation. I think that compressing only on flush is a big issue. It was a lot harder to code it, but it really makes a difference. You don't want a machine with a working set that fits into RAM to be compressing, that would be lethal to performance, and it is a very important case. Hans
Re: reiser4 status (correction)
David Masover wrote: And it's not just databases. Consider BitTorrent. The usual BitTorrent way of doing things is to create a sparse file, then fill it in randomly as you receive data. Only if you decide to allocate the whole file right away, instead of making it sparse, you gain nothing on Reiser4, since writes will be just as fragmented as if it was sparse. If you don't flush it before you fill it, then in reiser4 it does not matter if it starts out sparse. Personally, I'd rather leave it as sparse, but repack everything later. We have 3 levels of optimization: 1) at each modification, 2) at each flush, and 3) at each repack. Each of these operates on a different time scale, and all 3 are worthy of doing as right as we can. Now, the issue of where should airholes be? Why, that is the grand experiment that will start to happen in a few months. Nobody knows yet what defaults we should have, and whatever we choose, there will be some users who gain from explicit control of it. it must not be as trivial as I think it is. The problem is that there was a list of must dos, and this was just one of them. If reiser4 goes in, then fsync is the only thing in front of the repacker. The list has reduced in size a bunch. A much better approach in my opinion would be to have Reiser4 perform well in the majority of cases without the repacker, and sell the repacker to people who need that extra bit of performance. If I'm not mistaken this is actually Hans intent. Hans? Yes, that's the idea. Only sysadmins of large corps are likely to buy. We throw in service and support as well for those who purchase it. If I was making money, I would not do this, but I am not. I am not really willing to work a day job for the rest of my life supporting guys in Russia, it is only ok to do as a temporary measure. I am getting tired If Reiser4 does turn out to perform much worse over time, I would expect Hans would consider it a bug or design flaw and try to correct the problem however possible. I would want Reiser4 without a repacker to outperform all other filesystems. The problem with this fragmentation over time issue is that it is hard to tweak allocation, measure the effect, tweak again, etc. Not sure how to address it without a lot of work. Maybe we need to create some sort of condensed portage benchmark Hans Hans
Re: reiser4 status (correction)
On Fri, 2006-07-21 at 23:53 -0600, Hans Reiser wrote: We have 3 levels of optimization: 1) at each modification, 2) at each flush, and 3) at each repack. Each of these operates on a different time scale, and all 3 are worthy of doing as right as we can. Now, the issue of where should airholes be? Why, that is the grand experiment that will start to happen in a few months. Nobody knows yet what defaults we should have, and whatever we choose, there will be some users who gain from explicit control of it. Wouldn't the most recently modified files give a good hint to the repacker? The files larger then a few kilobytes most recently modified would be packed with air holes, and the files least recently modified would be packed more tightly? Perhaps even files that are currently the most fragmented would be repacked with air holes, as compared to files least fragmented would be packed more tightly. Could you not also write a small little app that gathers all kinds of stats about a file system and sends it to a Namesys server in hopes of finding better statistical data? I'm sure there are thousands of users that would be willing to run this app for the greater good regardless or not if they used ReiserFS in the first place. Things like the number of files on the disk, what percentage of those files have been modified in the last week, which files are the most/least fragmented, and when they were last modified, etc...? A much better approach in my opinion would be to have Reiser4 perform well in the majority of cases without the repacker, and sell the repacker to peopleisn' something who need that extra bit of performance. If I'm not mistaken this is actually Hans intent. Hans? Yes, that's the idea. Only sysadmins of large corps are likely to buy. We throw in service and support as well for those who purchase it. If I was making money, I would not do this, but I am not. I am not really willing to work a day job for the rest of my life supporting guys in Russia, it is only ok to do as a temporary measure. I am getting tired Personally, as much as I would like it all to be free, I think I would be much more willing to pay for compression/encryption (on both servers and desktops) then I would be for a repacker. Hard disks cost money, and if I can compress the vast majority of my data and save on purchasing a new hard disk, that is well worth it. I also have some important data that I would really like to encrypt, which is also worth spending money on. But gaining ~10% in performance probably isn't worthwhile spending money on as I most likely wouldn't notice a difference in my day to day life, unless my server was incredibly busy. There is no doubt there is a market for a repacker, but I think people are much more likely to spend money on something that is immediately tangible, like disk space instantly being free'd up by compression, or data instantly being encrypted. As compared to something that is much less tangible like fragmentation percentages and minor I/O throughput improvements. I used to work at a large, world wide web hosting company and I could see making a case to management for purchasing Reiser4 compression would be pretty easy for our shared servers. Instantly freeing up large amounts of disk space (where .html/.php files were the vast majority) would save huge amounts of money on disk drives, especially since most of the servers used RAID1 and adding new drives was a huge pain in the neck. Making a case to purchase a repacker would be much, much more difficult. See, customers who used lots of CPU were easy to up-sell to a dedicated server because page load times were tangible and if they didn't move we would be forced to shut them off. However customers who used gobs of disk space were much more difficult to up-sell to dedicated servers because it didn't affect themselves or other customers in a tangible way. They wouldn't notice any difference by moving to a much more expensive dedicated server. I would like to see Namesys succeed and become incredibly profitable for Hans, if nothing else for the fact that he has given a huge amount to the open source community already. A profitable Namesys only means we'll have a greater chance of seeing even more interesting stuff from them in the future. -- Mike Benoit [EMAIL PROTECTED] signature.asc Description: This is a digitally signed message part
Re: reiser4 status (correction)
Mike Benoit wrote: Could you not also write a small little app that gathers all kinds of stats about a file system and sends it to a Namesys server in hopes of finding better statistical data? I'm sure there are thousands of users Assuming the results are all made available, essentially public domain. If it becomes improve Reiser instead of improve filesystems, then only fans of Reiser will do it. A much better approach in my opinion would be to have Reiser4 perform well in the majority of cases without the repacker, and sell the repacker to peopleisn' something who need that extra bit of performance. If I'm not mistaken this is actually Hans intent. Hans? Yes, that's the idea. Only sysadmins of large corps are likely to buy. We throw in service and support as well for those who purchase it. Personally, as much as I would like it all to be free, I think I would be much more willing to pay for compression/encryption (on both servers and desktops) then I would be for a repacker. Hard disks cost money, and if I can compress the vast majority of my data and save on purchasing a new hard disk, that is well worth it. I also have some important data that I would really like to encrypt, which is also worth spending money on. But gaining ~10% in performance probably isn't worthwhile spending money on as I most likely wouldn't notice a difference in my day to day life, unless my server was incredibly busy. Assuming the performance gain is only 10%, you may be right. Still, faster disks, controllers, buses, and CPUs also cost money. I would be willing to pay for both, even, if the price was reasonable (I think there was a scheme based on amount of space placed under the FS?), and if I could be guaranteed updates (bug fixes, new kernels, etc) for at least the functionality that I've paid for, with no additional cost. Reiser4 lazy writes make a huge difference on a laptop, and my current laptop is a Mac. That means that around when the next Mac OS rolls out, it will be perfectly reasonable to spend some $50 or $100 to make my Linux faster instead of my Mac OS, especially since I'm trying to migrate off of Mac OS as much as possible. (I miss package management.) One more suggestion: Maybe make it free for non-commercial use only? And by that I mean, based on how the FS is actually being used. I don't know if piracy is or has been a problem for Namesys, but at least the economics of it makes sense: A fifteen year old hacker won't want to pay for an FS, but might have a lot to contribute. But if your main market is large servers, have those people pay -- they are running their business off your FS. There is no doubt there is a market for a repacker, but I think people are much more likely to spend money on something that is immediately tangible, like disk space instantly being free'd up by compression, or data instantly being encrypted. As compared to something that is much The compression will probably mostly be about speed. Remember, if we're talking about people who want to see tangible, visceral results, we're probably also talking about end-users. And trust me, the vast majority of most of my data (as an end-user) is not very compressible. Ok, I lied: I love games, and you can make a very modern-looking game in 96K: http://produkkt.abraxas-medien.de/kkrieger But while that could be seen as a kind of compression, it's done by hand, and is fairly irrelevant to filesystems. No, mostly we're talking about things like office documents, the majority of which fit in less than a gigabyte, and multimedia (music, movies, games) which will gain very little from compression. If anything, the benefit would be mostly in compressing software. less tangible like fragmentation percentages and minor I/O throughput improvements. I used to work at a large, world wide web hosting company and I could see making a case to management for purchasing Reiser4 compression would be pretty easy for our shared servers. Instantly freeing up large amounts of disk space (where .html/.php files were the vast majority) would save huge amounts of money on disk drives, especially since most of the servers used RAID1 and adding new drives was a huge pain in the neck. Making a case to purchase a repacker would be much, much more difficult. Hmm, the problem is, if storage space is really the big deal, it's been done before, and some of these efforts are still usable and free: http://parallel.vub.ac.be/~johan/compFUSEd/ http://www.miio.net/fusecompress/ http://north.one.pl/~kazik/pub/LZOlayer/ http://apfs.humorgraficojr.com/apfs_ingles.html And while we're on the topic, here's an FS that does unpacking of archives, probably about the same way we imagined it in Reiser4 pseudofiles/magic: http://www.nongnu.org/unpackfs/ But regardless, as far as I can tell, the only real, tangible benefit of using Reiser4 compression instead of one of those four FUSE filesystems is
portage tree (Was: Re: reiser4 status (correction))
Hi,
The portage tree is such a fine testing object since it should be sort
of a best case scenario for reiser filesystems, and needs no real
backup in case of a screwup during tests.
I've been on Gentoo for years now, used reiser3 since the days when you
had to patch it into a 2.2 kernel and tried out reiser4 on my portage
tree first of all. During my comparison, reiser3 + notail resulted in
huge amounts of wasted disk space, obviously it is not very smart to use
up a whole 4k block for a file of only few bytes. reiser3 with packed
tails was a lot better, but still it was a remarkable difference from
there to reiser4, and I am only talking storage efficiency here which
directly translates to the amount of I/O necessary to get the same data
all the way to RAM and CPU. The reiser4 dev team did a great job there!
With reiser3, I made another observation wrt. fragmentation, most of
which should be alleviated by reiser4 and its delayed allocation. Over
time, doing package database updates and querying for pending updates to
the local machine became slower. Packing up everything in a tarball,
mkfs, mount with -onoatime, unpack usually fixed the problem. Since
the performance reduction in reiser4 is a lot less (I can hardly tell if
it makes a difference before vs. after the backup/restore repacking) I
hardly do it any more, but once in a while I still use my script
initially written for the comparison test.
Attached is the shell script I used for comparing various mount options
for ReiserFS v3 and later v4. It is capable of converting between
ext2|3, reiser3|4 and xfs, provided the target fs stores the data
efficiently enough that everything fits inside. (I had interesting
results when moving from reiser3 to ext3 for testing, esp. wrt. portage
tree ; )
The script falls short at handling arbitrary mount options, but may be
trivially edited to use any desired options for a single use. Reading
and understanding is recommended before feeding your viable data to it.
I hope somebody may find it useful, or at least inspiring. Any comments
appreciated!
Kind regards,
Chris
#!/bin/sh
# ! WARNING !
# As always, BE CAREFUL AND KNOW WHAT YOU DO WHEN RUNNING THIS PROGRAM!
# At best, read the code, understand what it does, and take appropriate
# precautions. Backups of valuable data are a requirement anyway, so go
# for it BEFORE you try this out!
#
#
# SYNOPSIS:
# fsconvert.sh fstype compression method mountpoint
#
# where fstype may be any of ext[2|3], reiser[fs|4] or xfs
# and compression method is a choice of compress, gzip or bzip2.
#
#
# USE:
# This script was initially intended to re-create old reiserfs 3
# filesystems for performance reasons, and can reasonably be considered
# useful when run once every year or two for any filesystem under heavy
# read/write load. Simply give the existing filesystem type as the fstype
# argument.
#
# Though it was written for that single purpose at first, it was recently
# adapted for filesystem type migration kind of operations, as it did
# never care about the fs type in the beginning and now takes the
# destination fs type as an argument.
#
# The script REQUIRES the concerned fs to be mounted, to have an entry in
# /etc/fstab, to have no further active mountpoints in a subdirectory,
# enough free disk space in $archdir to back up data and of course the
# required tools (mkfs.fstype, tar, compression tool of choice) to do
# all the magic.
#
# Please send any comment, modification, failure report etc. to
# ctrefzer AT gmx DOT de
#
# Have fun!
# Chris
# Changelog:
#
# 2005-10-03
# Enhanced commandline interface to support multiple target file system types
# and compression methods.
# Filesystems supported so far:
# - ext2 / 3
# - reiser 3 / 4
# - xfs
#
# Compression methods which seemed appropriate:
# - none (uncompressed archive for almost no CPU load)
# - compress (quite fast even on old machines)
# - gzip (realtime on new machines, yet impressive I/O reduction)
# - bzip2 (in case of tight disk space, but takes ages wrt. the others)
### Configure here:
# where to store the archives:
archdir=${PWD}/.fsconvert
# Sane defaults for compression levels (and guidelines): use fast gzip on
# modern CPUs where GZIP -3 gives better compression than LZW, yet still at
# realtime, whereas BZIP2 should only be used when disk space is _really_
# critical - it will take forever. LZW (compress) is advised on not-so-recent
# machines to save some I/O without slowing things down. A nifty compromise
# may be GZIP -1 on P2/P3 class machines.
export GZIP=-1
export BZIP2=-9
### No modifications required below here!
### Some helper functions:
function e_report {
echo -en ${1}...
}
function e_done {
# Paranoia is good! It's all about the data...
sync
echo Done.
}
function e_fail {
echo FAILED!
exit ${1}
}
function e_usage {
echo USAGE: ${0} fstype compressiontype mountpoint
exit 1
}
if [ -z ${1} ]
then
Re: reiser4 status (correction)
On Sat, 2006-07-22 at 07:34 -0500, David Masover wrote: The compression will probably mostly be about speed. Remember, if we're talking about people who want to see tangible, visceral results, we're probably also talking about end-users. And trust me, the vast majority of most of my data (as an end-user) is not very compressible. Sure, mine too. Between the many gigs of MP3s and movies I have stored on my HD, only about 10-20GB is the OS, Email, and documents/source code. Even just compressing that small portion though I could probably save between 5-10GB. The difference is though I can do a df before, and a df after, and I can instantly see I got my moneys worth. Same with encryption. With the repacker it is much more difficult (for average users) to time how long a program takes to load some file (or launch), before the repacker and after. Especially since caching comes in to play. Also, according to this little poll on one of the compressed FUSE sites you linked to, more people are looking to compression for space saving, then for speed: http://parallel.vub.ac.be/~johan/compFUSEd/index.php?option=pollstask=resultspollid=31 No, mostly we're talking about things like office documents, the majority of which fit in less than a gigabyte, and multimedia (music, movies, games) which will gain very little from compression. If anything, the benefit would be mostly in compressing software. less tangible like fragmentation percentages and minor I/O throughput improvements. I used to work at a large, world wide web hosting company and I could see making a case to management for purchasing Reiser4 compression would be pretty easy for our shared servers. Instantly freeing up large amounts of disk space (where .html/.php files were the vast majority) would save huge amounts of money on disk drives, especially since most of the servers used RAID1 and adding new drives was a huge pain in the neck. Making a case to purchase a repacker would be much, much more difficult. Hmm, the problem is, if storage space is really the big deal, it's been done before, and some of these efforts are still usable and free: http://parallel.vub.ac.be/~johan/compFUSEd/ http://www.miio.net/fusecompress/ http://north.one.pl/~kazik/pub/LZOlayer/ http://apfs.humorgraficojr.com/apfs_ingles.html And while we're on the topic, here's an FS that does unpacking of archives, probably about the same way we imagined it in Reiser4 pseudofiles/magic: http://www.nongnu.org/unpackfs/ But regardless, as far as I can tell, the only real, tangible benefit of using Reiser4 compression instead of one of those four FUSE filesystems is speed. Reiser4 would compress/decompress when actually hitting the disk, not just the FS, and it would also probably use in-kernel compression, rather than calling out to userspace on every FS operation. But you see, if you're talking about speed, 10% is a respectably big improvement, so I could see selling them on a repacker at the same time. Most of those FUSE file systems you linked to scare me. This is from the APFS web page: Permissions do not work! Do not worry, it is not your fault, permissions are not implemented yet. It is like all files had rwxrwxrwx mode. I have lost all my data! How do I get it back? From the backup, obviously. FUSE is great, but can it even come close to matching the performance of in-kernel file systems? Not only that, but if you want to compress a directory you have to go through about a 12 step process of moving the files, setting up a mount point, and moving the files back. Will Reiser4 not allow us to mount with compression enabled then enable/disable compression on a per file/per directory basis? Maybe bundles are a good idea... Maybe there should be a Reiser4 Whitepaper Value Pack, once everything on the whitepaper is done? See, customers who used lots of CPU were easy to up-sell to a dedicated server because page load times were tangible and if they didn't move we would be forced to shut them off. However customers who used gobs of disk space were much more difficult to up-sell to dedicated servers because it didn't affect themselves or other customers in a tangible way. They wouldn't notice any difference by moving to a much more expensive dedicated server. Sounds like more a marketing problem than a technical one. Couldn't you just charge more on the virtual server? Or start charging by the megabyte? You could, and we did charge by the megabyte, only after they exceeded the limit of their package. However web hosting is a fiercely competitive market, so we were constantly adjusting our packages to include more disk space, more bandwidth, more features for the same amount, or lower prices, just to compete. The way many shared hosting companies work is each server is waaay over sold, at least in the disk space department. We would have anywhere from 500-1500 accounts
Re: reiser4 status (correction)
Hello Andreas, Saturday, July 22, 2006, 1:06:54 AM, you wrote: On 17:45 Fri 21 Jul , David Masover wrote: Question, then: Can the ext2 defrag work on a raw ext3 partition, without having to convert it first? Dunno, but I don't think so I tried that once, back in 2002 i think. defrag seemed doing things fine, but thrashed the filesystem very badly. The good thing is, that I was able to fsck it and recover most of my files. However it is not something you should try. -- Best regards, Maciej
Re: reiser4 status (correction)
Mike Benoit wrote: code. Even just compressing that small portion though I could probably save between 5-10GB. The difference is though I can do a df before, and a df after, and I can instantly see I got my moneys worth. Same with encryption. In the case of encryption, it's also got competition. There are two FUSE filesystems that do crypto, and there's cryptoloop/dm-crypt, which you need anyway for encrypted swap. True, it's nowhere near as nice, but it is functional. You would think one of these would be easier to develop to where it's useful. And if you're encrypting it, you're already accepting a performance hit. So again, the Reiser4 advantage here is all speed. http://parallel.vub.ac.be/~johan/compFUSEd/index.php?option=pollstask=resultspollid=31 Of course they are, or they wouldn't use FUSE. But if that's really true, why wouldn't a FUSE driver help you? Most of those FUSE file systems you linked to scare me. This is from the APFS web page: I have lost all my data! How do I get it back? From the backup, obviously. To be fair, that's what you get from any FS. But how hard would it be to make a FUSE filesystem work properly? How hard would it be to get a Reiser4 plugin to work properly? FUSE is great, but can it even come close to matching the performance of in-kernel file systems? Performance again! Not only that, but if you want to compress a directory you have to go through about a 12 step process of moving the files, setting up a mount point, and moving the files back. You listed 3 steps. Besides, I don't see anything stopping you from modifying one of these to selectively compress things, the way Reiser4 would. So, put your entire FS under a FUSE system, then configure which directories you want compressed. Again, the drawback is huge gobs of performance. It just seems natural to use the repacker if you're going to use any Reiser4-based replacement for these. You could, and we did charge by the megabyte, only after they exceeded the limit of their package. However web hosting is a fiercely competitive market, so we were constantly adjusting our packages to include more disk space, more bandwidth, more features for the same amount, or lower prices, just to compete. The way many shared hosting companies work is each server is waaay over sold, at least in the disk Ah, so I guess the advantage of a more expensive, dedicated box is that you know no one's overselling it? Personally, I get a little sick of the insane amounts of overselling that happen -- you just know it's going to come back and bite you in the ass. This is happening right now with bandwidth, and overselling is pretty much solely responsible for the whole Net Neutrality mess -- it would be a complete non-issue if 5 megabits to your house! was backed up by an actual 5 megabits reserved for me, or if they sold them as 2-5 megabits, where 2 is guaranteed, and 5 is what you get when no one else is using it. I'm a bit skeptical of my local ISP's Fiber To The Home initiative, because I can't imagine they have enough spare upstream bandwidth just lying around. I mean, if you actually have enough bandwidth, you don't want or need your ISP to do QoS or prioritizing for you -- you can just do it yourself. Personally, ssh packets would be of a much higher priority to me than anything else... Thats pretty much the only way you make money with $10-20/month packages. Yeah, I know, hard for an honest guy to compete... I don't doubt the benefits of the repacker, but from a business perspective the repacker is something that runs transparently in the background, once you install it, things magically speed up then you never hear from it again as it does its job. Out of sight, out of mind. Whereas the compression/encryption plugin are always in your face, every time you run df, or enter a passphrase to gain access to your files, you know its there and working. Its something you'll tell your friends about. After you first run it, the repacker just fades away in to the background and you forget about it. Ah. This might explain the success of things like Ruby On Rails. Once it's done, the insane amount of CPU required to run a high-level interpreted language (versus even something semi-interpreted like perl) is out of sight, out of mind. But when you first set it up, the savings in development time are immediately obvious, in your face. Although I would want to find a way to avoid typing a passphrase every time -- maybe keep the key on a USB keychain. Passphrases would just annoy users. Unless, of course, you could tie it to logon -- assuming you can actually change the passphrase later... Charging only for commercial use of the repacker/compression/encryption plugin would be a great middle ground. Good, I'm glad it wasn't a completely insane idea.
Re: reiser4 status (correction)
David Masover wrote: Hans Reiser wrote: On a more positive note, Reiser4.1 is getting closer to release Good news! But it's been awhile since I've followed development, and the homepage seems out of date (as usual). Where can I find a list of changes since v4? By out of date, I mean things like this: Reiser4.1 will modify the repacker to insert controlled air holes, as it is well known that insertion efficiency is harmed by overly tight packing. Sigh, no, the repacker will probably be after 4.1 The list of tasks for zam looks something like: fix bugs that arise debug read optimization code (CPU reduction only, has no effect on IO), 1 week est. (would be nice if it was less) review compression code 1 day per week until it ships. fix fsync performance (est. 1 week of time to make post-commit writes asynchronous, maybe 3 weeks to create fixed-reserve for write twice blocks, and make all fsync blocks write twice) write repacker (12 weeks). I am not sure that putting the repacker after fsync is the right choice The task list for vs looks like: * fix bugs as they arise. * fix whatever lkml complains about that either seems reasonable, or that akpm agrees with. * Help edward get the compression plugins out the door. * Improve fsck's time performance. * Fix any V3 bugs that Chris and Jeff don't fix for us. Which reminds me, I need to check on whether the 90% full bug got fixed
Re: reiser4 status (correction)
Sigh, no, the repacker will probably be after 4.1
The list of tasks for zam looks something like:
fix bugs that arise
debug read optimization code (CPU reduction only, has no effect on IO),
1 week est. (would be nice if it was less)
review compression code 1 day per week until it ships.
fix fsync performance (est. 1 week of time to make post-commit writes
asynchronous, maybe 3 weeks to create fixed-reserve for write twice
blocks, and make all fsync blocks write twice)
write repacker (12 weeks).
Well, this is free software, and is not backed u with a million dollars of
funding. Trust me, these guys are doing a great job with the options they
have.
This is not a fliambait, but I personally feel that this is way too great
considering the resources that the entire team has. I am more than willing to
wait even though I have had occasional bad experiences with reiser{3,4}.
The bottemline is that all versions of reiserfs have had great performance,
even though with its own hiccups (which fs doesn't have ??) and simply put,
it is a fantastic peice of code.
--
My mother loved children -- she would have given anything if I had been one.
-- Groucho Marx
Re: reiser4 status (correction)
Hans Reiser wrote:
I am not sure that putting the repacker after fsync is the right choice
Does the repacker use fsync? I wouldn't expect it to.
Does fsync benefit from a properly packed FS? Probably.
Also, while I don't expect anyone else to be so bold, there is a way
around fsync performance: Disable it. Patch the kernel so that any
fsync call from userspace gets ignored, but lie and tell them it worked.
Basically:
asmlinkage long sys_fsync(unsigned int fd)
{
- return do_fsync(fd, 0);
+ return 0; // do_fsync(fd, 0);
}
In order to make this sane, you should have backups and an Uninterrupted
Power Supply. In the case of a loss of power, the box should notice and
immediately sync, then either shut down or software suspend. Any UPS
battery should be able to handle the amount of time it takes to shut the
system off.
Since anything mission critical should have backups and a UPS anyway,
the only problem left is what happens if the system crashes. But system
crashes are something you have to plan for anyway. Disks fail -- stuff
happens. RAID won't save you -- the RAID controller itself will
eventually fail.
So suppose you're running some very critical server -- for now, chances
are it's running some sort of database. In this case, what you really
want is database replication. Have at least two servers up and running,
and consider the transaction complete not when it hits the disk, but
when all running servers acknowledge the transaction. The RAM of two
boxes should be safer than the disk of one.
What about a repacker? The best I can do to hack around that is to
restore the whole box from backup every now and then, but this requires
the box to be down for awhile -- it's a repacker, but not an online one.
In this case, the solution would be to have the same two servers
(replicating databases), and bring first one down, and then the other.
That would make me much more nervous than disabling fsync, though,
because now you only have the one server running, and if it goes down...
And depending on the size of the data in question, this may not be
feasable. It seems entirely possible that in some setups like this, the
only backup you'd be able to afford would be in the form of replication.
In my own personal case, I'd prefer the repacker to tuning fsync. But
arguments could be made for both.
Re: reiser4 status (correction)
On Fri, 2006-07-21 at 02:44 -0600, Hans Reiser wrote: fix fsync performance (est. 1 week of time to make post-commit writes asynchronous, maybe 3 weeks to create fixed-reserve for write twice blocks, and make all fsync blocks write twice) write repacker (12 weeks). I am not sure that putting the repacker after fsync is the right choice Tuning fsync will fix the last wart on Reiser4 as far as benchmarks are concerned won't it? Right now Reiser4 looks excellent on the benchmarks that don't use fsync often (mongo?), but last I recall the fsync performance was so poor it overshadows the rest of the performance. It would also probably be more useful to a much wider audience, especially if Namesys decides to charge for the repacker. ReiserV3 is used on a lot of mail and squid proxy servers that deal with many small files, and these work loads usually call fsync often. My guess is that ReiserV3 users are the most likely to migrate to Reiser4, because they already know the benefits of using a Reiser file system. But neglecting fsync performance will just put a sour taste in their mouth. On top of that, I don't see how a repacker would help these work loads much as the files usually have a high churn rate. Packing them would probably be a net loss as the files would just be deleted in 24hrs and replaced by new ones. Very few people will (or should) disable fsync as David suggests, I don't see that as a solution at all, even if it is temporary. -- Mike Benoit [EMAIL PROTECTED] signature.asc Description: This is a digitally signed message part
Re: reiser4 status (correction)
Mike Benoit wrote: Tuning fsync will fix the last wart on Reiser4 as far as benchmarks are concerned won't it? Right now Reiser4 looks excellent on the benchmarks that don't use fsync often (mongo?), but last I recall the fsync performance was so poor it overshadows the rest of the performance. It would also probably be more useful to a much wider audience, especially if Namesys decides to charge for the repacker. If Namesys does decide to charge for the repacker, I'll have to consider whether it's worth it to pay for it or to use XFS instead. Reiser4 tends to become much more fragmented than most other Linux FSes -- purely subjective, but probably true. ReiserV3 is used on a lot of mail and squid proxy servers that deal with many small files, and these work loads usually call fsync often. [...] But neglecting fsync performance will just put a sour taste in their mouth. So will neglecting fragmentation, only worse. At least fsync is slow up front. Fragmentation will be slow much farther in, when the mailserver has already been through one painful upgrade. Charging for the repacker just makes it worse. On top of that, I don't see how a repacker would help these work loads much as the files usually have a high churn rate. Packing them would probably be a net loss as the files would just be deleted in 24hrs and replaced by new ones. Depends. Some will, some won't. My IMAP server does have a lot of churning, but there's also the logs (which stay for at least a month or two before they rotate out), and since it's IMAP, I do leave quite a lot of files alone. v3 is also used on a lot of web servers, at least where I used to work -- some areas will be changing quite a lot, and some areas not at all. Changing a lot means fragmentation will happen, not changing at all means repacking will help. These issues may be helped by partitioning, if you know how you're going to split things up. But then, how do you partition in the middle of a squid server? A lot of people visit the same sites every day, checking for news, but that means plenty of logos, scripts, and other things won't change -- but plenty of news articles will change every couple hours. Very few people will (or should) disable fsync as David suggests, I don't see that as a solution at all, even if it is temporary. I guess the temporary solution is to incur a pretty big performance hit. But it comes back to, which is more of a performance problem, fsync or fragmentation? And I really would like to hear a good counter-argument to the one I've given for disabling fsync. But even if we assume fsync must stay, do we have any benchmarks on fragmentation versus fsync? But maybe it's best to stop debating, since both will be done eventually, right?
Re: reiser4 status (correction)
On Fri, 2006-07-21 at 16:06 -0500, David Masover wrote: Mike Benoit wrote: Tuning fsync will fix the last wart on Reiser4 as far as benchmarks are concerned won't it? Right now Reiser4 looks excellent on the benchmarks that don't use fsync often (mongo?), but last I recall the fsync performance was so poor it overshadows the rest of the performance. It would also probably be more useful to a much wider audience, especially if Namesys decides to charge for the repacker. If Namesys does decide to charge for the repacker, I'll have to consider whether it's worth it to pay for it or to use XFS instead. Reiser4 tends to become much more fragmented than most other Linux FSes -- purely subjective, but probably true. I would like to see some actual data on this. I haven't used Reiser4 for over a year, and when I did it was only to benchmark it. But Reiser4 allocates on flush, so in theory this should decrease fragmentation, not increase it. Due to this I question what you are _really_ seeing, or if perhaps it is a bug in the allocator? Why would XFS or any other multi-purpose file system resist fragmentation noticeably more then Reiser4 does. I don't think the repacker is designed to be a must have for every Reiser4 installation. If it was, I would consider Reiser4 to be seriously flawed. Instead I think it is simply designed to improve certain workloads that may cause high fragmentation in hopes of keeping I/O speeds at their peek. Am I correct in this assumption Hans? No Linux file system that I'm aware of has a defragmentor, but they DO become fragmented, just not near as bad as FAT32 used to when MS created their defragmentor. The highest non-contiguous percent I've seen with EXT3 is about 12%, FAT32 I have seen over 50%, and NTFS over 30%. In fact I'm running in to a fragmentation issue with ReiserV3 right now that Jeff is working on, but it is more of a worst case scenario issue, not a regular occurrence issue. For normal workloads I doubt you would notice much difference at all by using a repacker, 10% maybe? Which is one of the reasons you probably haven't seen a repacker for EXT2/3, even though I'm sure it would improve performance for some people. -- Mike Benoit [EMAIL PROTECTED] signature.asc Description: This is a digitally signed message part
Re: reiser4 status (correction)
On 14:37 Fri 21 Jul , Mike Benoit wrote: No Linux file system that I'm aware of has a defragmentor, but they DO become fragmented, just not near as bad as FAT32 used to when MS created their defragmentor. Forgotten ext2? ;-) Funny thing: If your ext3 got too fragmented: convert it back to ext2, defrag and reconvert it again to ext3. All of this can be done in place, i.e. without moving the data to other partitions etc. -Andreas pgpiwx2SW0fdR.pgp Description: PGP signature
Re: reiser4 status (correction)
Mike Benoit wrote: On Fri, 2006-07-21 at 16:06 -0500, David Masover wrote: Mike Benoit wrote: Tuning fsync will fix the last wart on Reiser4 as far as benchmarks are concerned won't it? Right now Reiser4 looks excellent on the benchmarks that don't use fsync often (mongo?), but last I recall the fsync performance was so poor it overshadows the rest of the performance. It would also probably be more useful to a much wider audience, especially if Namesys decides to charge for the repacker. If Namesys does decide to charge for the repacker, I'll have to consider whether it's worth it to pay for it or to use XFS instead. Reiser4 tends to become much more fragmented than most other Linux FSes -- purely subjective, but probably true. I would like to see some actual data on this. I haven't used Reiser4 for over a year, and when I did it was only to benchmark it. But Reiser4 allocates on flush, so in theory this should decrease fragmentation, not increase it. Due to this I question what you are _really_ seeing, or if perhaps it is a bug in the allocator? Why would XFS or any other multi-purpose file system resist fragmentation noticeably more then Reiser4 does. Maybe not XFS, but in any case, Reiser4 fragments more because of how its journaling works. It's the wandering logs. Basically, when most Linux filesystems allocate space, they do try to allocate it contiguously, and it generally stays in the same place. With ext3, if you write to the middle of a file, or overwrite the entire file, you're generally going to see your writes be written once to the journal, and then again to the same place the file originally was. Similarly, if you delete and then create a bunch of small files, you're generally going to see the new files created in the same place the old files were. With Reiser4, wandering logs means that rather than write to the journal, if you write to the middle of the file, it writes that chunk to somewhere else on the disk, and somehow gets it down to one atomic operation where it simply changes the file to point to the new location on disk. Which means if you have a filesystem that is physically laid out on disk like this (for simplicity, assume it only has a single file): # is data * is also data - is free space ##*-- When you try to write in the middle (the '*' chars) -- let's say we're changing them to '%' chars, this happens: ##*%- Once that's done, the file is updated so that the middle of it points to the fragment in the new location, and the old location is freed: ##-%- Keep in mind, because of lazy writes, it's much more likely for the whole change to happen at once. Here's another example: # Let's say we just want to overwrite the file with another one of the same length: #%--- then, commit the transaction: -%--- You see the problem? You've now split the free space in half. Realistically, of course, it wouldn't be by halves, but you're basically inserting random air holes all over the place, and your FS is becoming more like foam, taking up more of the free space, until you can no longer use the free space In the above example, if we then have to come write some huge file, it looks like this: *%*** Split right in half. Now imagine this effect multiplied by hundreds or thousands of files, over time... This is why Reiser4 needs a repacker. While it's fine for larger files -- I believe after a certain point, it will write twice, so looking at our first example: ##*-- Write to a new, temporary place: ##*%- Write back to the original place: ##%%- Complete the transaction and free the temporary space: ##%-- This technique is what other journaling filesystems use, and it also means that writing is literally twice as slow as on a non-journaling filesystem, or on one with wandering logs like Reiser4. But, it's a practical necessity when you're dealing with some 300 gig MySQL database of which only small 10k chunks are changing. Taking twice as long on a 10k chunk won't kill anyone, but fragmenting your 300 gig database (on a 320 gig partition) will kill your performance, and will be very difficult to defragment. But on smaller files, it would be very beneficial if we could allow the FS to slowly fragment (to foam-ify, if you will) and defrag once a week. The amount of speed gained in each write -- and read, if it's not getting too awful during that week -- definitely makes up for having to spend an hour or so defragmenting, especially if the FS can be online at the time. And you can probably figure out an optimal time to wait before defragmenting, since your biggest fragmentation problems happen when the chunk of contiguous space at the end of the
Re: reiser4 status (correction)
Andreas Schäfer wrote: On 14:37 Fri 21 Jul , Mike Benoit wrote: No Linux file system that I'm aware of has a defragmentor, but they DO become fragmented, just not near as bad as FAT32 used to when MS created their defragmentor. Forgotten ext2? ;-) Funny thing: If your ext3 got too fragmented: convert it back to ext2, defrag and reconvert it again to ext3. All of this can be done in place, i.e. without moving the data to other partitions etc. Well, I know an ext3 partition can be mounted, unchanged, as ext2. Of course, you had to cleanly unmount as ext3 first, and make sure you cleanly mount/unmount as ext2 before you try to mount as ext3 again... Question, then: Can the ext2 defrag work on a raw ext3 partition, without having to convert it first?
Re: reiser4 status (correction)
On 17:45 Fri 21 Jul , David Masover wrote: Question, then: Can the ext2 defrag work on a raw ext3 partition, without having to convert it first? Dunno, but I don't think so pgp3cS9WVSQCi.pgp Description: PGP signature
Re: reiser4 status (correction)
Your detailed explanation is appreciated David and while I'm far from a file system expert, I believe you've overstated the negative effects somewhat. It sounds to me like you've gotten Reiser4's allocation process in regards to wandering logs correct, from what I've read anyways, but I think you've overstated its fragmentation disadvantage when compared against other file systems. I think the thing we need to keep in mind here is that fragmentation isn't always a net loss. Depending on the workload, fragmentation (or at least not tightly packing data) could actually be a gain. In cases where you have files (like log files or database files) that constantly grow over a long period of time, packing them tightly at regularly scheduled intervals (or at all?) could cause more harm then good. Consider this scenario of two MySQL tables having rows inserted to each one simultaneously, and lets also assume that the two tables were tightly packed before we started the insert process. 1 = Data for Table1 2 = Data for Table2 Tightly packed: Simultaneous inserts start: 1122112211221122 I believe this is actually what is happening to me with ReiserV3 on my MythTV box. I have two recordings running at the same time, each writing data at about 500kb/s and once the drive has less then 10% free the whole machine grinds to a screeching halt while it attempts to find free space. The entire 280GB drive is full of files fragmented like this. Allocate on flush alone would probably help this scenario immensely. The other thing you need to keep in mind is that database files are like their own little mini-file system. They have their own fragmentation issues to deal with (especially PostgreSQL). So in cases like you described where you are overwriting data in the middle of a file, Reiser4 may be poor at doing this specific operation compared to other file systems, but just because you overwrite a row that appears to be in the middle of a table doesn't mean that the data itself is actually in the middle of the table. If your original row is 1K, and you try to overwrite it with 4K of data, it most likely will be put at the end of the file anyways, and the original 1K of data will be marked for overwriting later on. Isn't this what myisampack is for? So while I think what you described is ultimately correct, I believe extreme negative effects from it to be a corner case, and probably not representative of the norm. I also believe that other Reiser4 improvements would outweigh this draw back to wandering logs, again in average workloads. So the original point that I was trying to make comes back to the fact that I don't believe Reiser4 _needs_ a repacker to maintain decent performance. The fact that it will have a repacker just makes it that much better for people who might need it. If Hans didn't think he could make money off it, it probably wouldn't be so high on his priority list? We can't fault him for that though. Like you mentioned, if Reiser4 performance gets so poor without the repacker, and Hans decides to charge for it, I think that will turn away a lot potential users as they could feel that this is a type of extortion. Get them hooked on something that only performs well for a certain amount of time, then charge them money to keep it up. I also think the community would write their own repacker pretty quick in response. A much better approach in my opinion would be to have Reiser4 perform well in the majority of cases without the repacker, and sell the repacker to people who need that extra bit of performance. If I'm not mistaken this is actually Hans intent. If Reiser4 does turn out to perform much worse over time, I would expect Hans would consider it a bug or design flaw and try to correct the problem however possible. But I guess only time will tell if this is true or not. ;) On Fri, 2006-07-21 at 17:40 -0500, David Masover wrote: Maybe not XFS, but in any case, Reiser4 fragments more because of how its journaling works. It's the wandering logs. Basically, when most Linux filesystems allocate space, they do try to allocate it contiguously, and it generally stays in the same place. With ext3, if you write to the middle of a file, or overwrite the entire file, you're generally going to see your writes be written once to the journal, and then again to the same place the file originally was. Similarly, if you delete and then create a bunch of small files, you're generally going to see the new files created in the same place the old files were. With Reiser4, wandering logs means that rather than write to the journal, if you write to the middle of the file, it writes that chunk to somewhere else on the disk, and somehow gets it down to one atomic operation where it simply changes the file to point to the new location on disk. Which means if you have a filesystem that is
Re: reiser4 status (correction)
Mike Benoit wrote: On top of that, I don't see how a repacker would help these work loads much as the files usually have a high churn rate. I think Reiserfs is used on a lot more than squid servers. For them, 80% of files don't move for long periods of time is the usual industry statistic
Re: reiser4 status (correction)
Mike Benoit wrote: Your detailed explanation is appreciated David and while I'm far from a file system expert, I believe you've overstated the negative effects somewhat. It sounds to me like you've gotten Reiser4's allocation process in regards to wandering logs correct, from what I've read anyways, but I think you've overstated its fragmentation disadvantage when compared against other file systems. I think the thing we need to keep in mind here is that fragmentation isn't always a net loss. Depending on the workload, fragmentation (or at least not tightly packing data) could actually be a gain. In cases where defragmented != tightly packed. you have files (like log files or database files) that constantly grow over a long period of time, packing them tightly at regularly scheduled intervals (or at all?) could cause more harm then good. This is true... Consider this scenario of two MySQL tables having rows inserted to each one simultaneously, and lets also assume that the two tables were tightly packed before we started the insert process. 1 = Data for Table1 2 = Data for Table2 Tightly packed: Simultaneous inserts start: 1122112211221122 Allocate on flush alone would probably help this scenario immensely. Yes, it would. You'd end up with assuming they both fit into RAM. And of course they could later be repacked. By the way, this is the NTFS approach to avoiding fragmentation -- try to avoid fragmenting anything below a certain block size. I, for one, would be perfectly happy if my large files were split up every 50 or 100 megs or so. The problem is when you get tons of tiny files and metadata stored so horribly inefficiently that things like Native Command Queuing is actually a huge performance boost. The other thing you need to keep in mind is that database files are like their own little mini-file system. They have their own fragmentation issues to deal with (especially PostgreSQL). I'd rather not add to that. This is one reason to hate virtualization, by the way -- it's bad enough to have a fragmented NTFS on your Windows installation, but worse if the disk itself is a fragmented sparse file on Linux. So in cases like you described where you are overwriting data in the middle of a file, Reiser4 may be poor at doing this specific operation compared to other file systems, but just because you overwrite a row that appears to be in the middle of a table doesn't mean that the data itself is actually in the middle of the table. If your original row is 1K, and you try to overwrite it with 4K of data, it most likely will be put at the end of the file anyways, and the original 1K of data will be marked for overwriting later on. Isn't this what myisampack is for? If what you say is true, isn't myisampack also an issue here? Surely it doesn't write out an entirely separate copy of the file? Anyway, the most common usage I can see for mysql would be overwriting a 1K row with another 1K row, or dropping a row, or adding a wholly new row. I may be a bit naive here... But then, isn't there also some metadata somewhere which says things like how many rows you have in a given table? And it's not just databases. Consider BitTorrent. The usual BitTorrent way of doing things is to create a sparse file, then fill it in randomly as you receive data. Only if you decide to allocate the whole file right away, instead of making it sparse, you gain nothing on Reiser4, since writes will be just as fragmented as if it was sparse. Personally, I'd rather leave it as sparse, but repack everything later. So while I think what you described is ultimately correct, I believe extreme negative effects from it to be a corner case, and probably not representative of the norm. I also believe that other Reiser4 improvements would outweigh this draw back to wandering logs, again in average workloads. Depends on your definition of average. I'm also speaking from experience. On Gentoo, /usr/portage started out being insanely fast on Reiser4, because it barely had to seek at all -- despite being about 145,000 small files. I think it was maybe half that when I first put it on r4, but it's more than twice as slow now, and you can hear it thrashing. Now, the wandering logs did make the rsync process pretty fast -- the entire thing gets rsync'd against one of the Gentoo mirrors. For anyone using Debian, this is the equivalent of apt-get update. Only now, this rsync process is not only entirely disk-bound, it's something like 10x as slow. I have a gig of RAM, so at least it's fast once it's cached, but it's obviously horrendously fragmented. I am not sure if it's individual files or directories, but it could REALLY use a repack. From what I remember of v3, it was never quite this bad, but it never started
Cannot display some characters anymore [reiser4-2.6.17-1] (was Re: reiser4 for 2.6.16 (version 3))
Hi, some time ago I reported an issue with characters not being displayed. This problem with netbeans (java) still persists with the latest patch reiser4-2.6.17-1 Thx for your work. Greetz Frieder Original Message Date: Tue, 06 Jun 2006 01:15:25 +0200 From: Frieder Bürzele [EMAIL PROTECTED] To: reiserfs-list@namesys.com Subject: Re: reiser4 for 2.6.16 (version 3) Vladimir V. Saveliev wrote: Hello ftp://ftp.namesys.com/pub/reiser4-for-2.6/2.6.16/reiser4-for-2.6.16-3.patch.gz contains the most recent reiser4 code which is considered stable inside Namesys. This code is supposed to be in mm kernel next to 2.6.17-rc4-mm3. Please try it. Any feedback is welcome. Hi, so far reiser4-for-2.6.16-[3|4].patch.gz runs great here, but I've recongised that when building a java-project with netbeans I got this output in the netbeans message-window: init: ��� � --snip-- instead of what I was used to see before with reiser4-for-2.6.16-2.patch.gz: init: deps-jar: compile: To run this application from the command line without Ant, try: --snip-- This seems very weird to me. I can't imagine that this could be issued by the filesystem so I just build clean kernels with the working and non-working patch. But problem still persist. system info: (default-linux/x86/2005.1, gcc-4.1.0, glibc-2.3.6-r4, 2.6.16 i686) = System uname: 2.6.16 i686 AMD Athlon(tm) XP 2100+ Gentoo Base System version 1.6.14 Sun JDK 1.5.0.05 [sun-jdk-1.5] netbeans-5.0 thx Frieder
Re: reiser4 status (correction)
Hans Reiser wrote: On a more positive note, Reiser4.1 is getting closer to release Good news! But it's been awhile since I've followed development, and the homepage seems out of date (as usual). Where can I find a list of changes since v4? By out of date, I mean things like this: Reiser4.1 will modify the repacker to insert controlled air holes, as it is well known that insertion efficiency is harmed by overly tight packing. Am I to understand that this text hasn't changed since the repacker was essentially canceled/removed/commercialized? Or is there actually a repacker somewhere usable, today, that is still scheduled to insert air holes by v4.1? Removed LKML from CC for now, for no particular reason. Add them back in, if you like.
Re: reiser4 OOPSes and panics when out of memory
Thanks. Now with debug enabled I've gotten: http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060719/panic1.txt.gz http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060719/fsck1_--check.txt.gz http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060719/fsck1_--fix.txt.gz http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060719/fsck1_--check_after_--fix.txt.gz and http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060719/messages2.txt.gz followed by http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060719/messages2b.txt.gz and without debug: http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060719/messages3.txt.gz http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060719/fsck3_--check.txt.gz http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060719/fsck3_--fix.txt.gz http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060719/fsck3_--check_after_--fix.txt.gz On Tue, 2006-07-18 at 18:18 +0400, Vladimir V. Saveliev wrote: Hello On Tue, 2006-07-18 at 00:52 -0600, Jake Maciejewski wrote: Thanks for the patch, but I can still reproduce the problem. I've been running the attached program to try to speed up the testing process a bit. Interrupting and restarting the compilation loop also seems to help. ok If I had hours to wait, it would probably crash eventually without additional encouragement, but I'm doing everything as an unprivileged user, so I don't think my tests are unreasonable. Anyway, I'm still getting a panic with debug enabled: reiser4 panicked cowardly: reiser4[find(16411)]: reiser4_dirty_inode (fs/reiser4/super_ops.c:173)[]: Kernel panic - not syncing: reiser4[find(16411)]: reiser4_dirty_inode (fs/reiser4/super_ops.c:173)[]: The attached patch should fix the above. Without debug enabled I've seen: http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060718/messages1.txt.gz http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060718/fsck1_--check.txt.gz http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060718/fsck1_--fix.txt.gz http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060718/fsck1_--check_after_--fix.txt.gz but usually I get: http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060718/messages3.txt.gz with no corruption (although I've been rebooting before complete failure). On Mon, 2006-07-17 at 21:38 +0400, Vladimir V. Saveliev wrote: Hello On Mon, 2006-07-17 at 18:10 +0400, Vladimir V. Saveliev wrote: Hello On Sun, 2006-07-16 at 12:44 -0500, [EMAIL PROTECTED] wrote: Has my previous post (http://marc.theaimsgroup.com/?l=reiserfsm=115259665831650w=2) been overlooked, or have I not provided enough information? Do I need to reproduce these issues on 2.6.18-rc1-mm2? Should I be trying any patches? please try the attached patch. your test crashes reiser4 on my test box. I hope to get a patch ready later today. Not sure that I got the same problem as you, though. We will see. The bottom line is with 2.6.17-mm6, I've always been able to OOPs or panic reiser4 on my amd64 machine (haven't tried x86 yet) by using all available physical memory. -- Jake Maciejewski [EMAIL PROTECTED]
Re: reiser4 OOPSes and panics when out of memory
Hello
On Wed, 2006-07-19 at 07:28 -0600, Jake Maciejewski wrote:
Thanks. Now with debug enabled I've gotten:
http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060719/panic1.txt.gz
the attached patch fixes a problem nikita-2967 reports about. Would you
please check whther it helps.
http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060719/fsck1_--check.txt.gz
http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060719/fsck1_--fix.txt.gz
http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060719/fsck1_--check_after_--fix.txt.gz
and
http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060719/messages2.txt.gz
followed by
http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060719/messages2b.txt.gz
and without debug:
http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060719/messages3.txt.gz
http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060719/fsck3_--check.txt.gz
http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060719/fsck3_--fix.txt.gz
http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060719/fsck3_--check_after_--fix.txt.gz
On Tue, 2006-07-18 at 18:18 +0400, Vladimir V. Saveliev wrote:
Hello
On Tue, 2006-07-18 at 00:52 -0600, Jake Maciejewski wrote:
Thanks for the patch, but I can still reproduce the problem. I've been
running the attached program to try to speed up the testing process a
bit. Interrupting and restarting the compilation loop also seems to
help.
ok
If I had hours to wait, it would probably crash eventually without
additional encouragement, but I'm doing everything as an unprivileged
user, so I don't think my tests are unreasonable.
Anyway, I'm still getting a panic with debug enabled:
reiser4 panicked cowardly: reiser4[find(16411)]: reiser4_dirty_inode
(fs/reiser4/super_ops.c:173)[]:
Kernel panic - not syncing: reiser4[find(16411)]: reiser4_dirty_inode
(fs/reiser4/super_ops.c:173)[]:
The attached patch should fix the above.
Without debug enabled I've seen:
http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060718/messages1.txt.gz
http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060718/fsck1_--check.txt.gz
http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060718/fsck1_--fix.txt.gz
http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060718/fsck1_--check_after_--fix.txt.gz
but usually I get:
http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060718/messages3.txt.gz
with no corruption (although I've been rebooting before complete
failure).
On Mon, 2006-07-17 at 21:38 +0400, Vladimir V. Saveliev wrote:
Hello
On Mon, 2006-07-17 at 18:10 +0400, Vladimir V. Saveliev wrote:
Hello
On Sun, 2006-07-16 at 12:44 -0500, [EMAIL PROTECTED] wrote:
Has my previous post
(http://marc.theaimsgroup.com/?l=reiserfsm=115259665831650w=2)
been
overlooked, or have I not provided enough information? Do I need to
reproduce these issues on 2.6.18-rc1-mm2? Should I be trying any
patches?
please try the attached patch.
your test crashes reiser4 on my test box. I hope to get a patch ready
later today. Not sure that I got the same problem as you, though. We
will see.
The bottom line is with 2.6.17-mm6, I've always been able to OOPs
or panic
reiser4 on my amd64 machine (haven't tried x86 yet) by using all
available
physical memory.
diff -puN fs/reiser4/plugin/file/file.c~reiser4-add-missing-unlock fs/reiser4/plugin/file/file.c
--- linux-2.6.17-mm6/fs/reiser4/plugin/file/file.c~reiser4-add-missing-unlock 2006-07-19 18:12:14.0 +0400
+++ linux-2.6.17-mm6-vs/fs/reiser4/plugin/file/file.c 2006-07-19 18:13:50.0 +0400
@@ -1636,14 +1636,18 @@ static size_t read_file(hint_t * hint, s
/* error happened */
break;
- if (coord-between != AT_UNIT)
+ if (coord-between != AT_UNIT) {
/* there were no items corresponding to given offset */
+ done_lh(hint-ext_coord.lh);
break;
+ }
loaded = coord-node;
result = zload(loaded);
- if (unlikely(result))
+ if (unlikely(result)) {
+ done_lh(hint-ext_coord.lh);
break;
+ }
if (hint-ext_coord.valid == 0)
validate_extended_coord(hint-ext_coord,
_
Re: reiser4 OOPSes and panics when out of memory
I haven't hit nikita-2967 again, but I got several other interesting results. The first panic didn't cause corruption: reiser4 panicked cowardly: reiser4[pdflush(16048)]: scan_by_coord (fs/reiser4/flush.c:3431)[nikita-3435]: Kernel panic - not syncing: reiser4[pdflush(16048)]: scan_by_coord (fs/reiser4/flush.c:3431)[nikita-3435]: The second affected my root partition, not the one I was stress testing: reiser4 panicked cowardly: reiser4[ent:hda3!(841)]: capture_anonymous_pages (fs/reiser4/plugin/file/file.c:1007)[vs-49]: Kernel panic - not syncing: reiser4[end:hda3!(841)]: capture_anonymous_pages (fs/reiser4/plugin/file/file.c:1007)[vs-49]: I booted from a live CD to document the corruption (which seemed to have been completely fixed). http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060720/fsck2_--check_hda3.txt.gz http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060720/fsck2_--fix_hda3.txt.gz http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060720/fsck2_--check_after_--fix_hda3.txt.gz When I rebooted, I got another panic when my system tried to mount / read-write: reiser4 pnicked cowardly: reiser4[mount(3614)]: check_blocks_bitmap (fs/reiser4/plugin/space/bitmap.c:1268)[zam-623]: Kernel panic - not syncing: reiser4[mount(3614)]: check_blocks_bitmap (fs/reiser4/plugin/space/bitmap.c:1268)[zam-623]: On the second reboot, it worked again. The third panic was one I've seen before (http://marc.theaimsgroup.com/?l=reiserfsm=115259665831650w=2): reiser4 panicked cowardly: reiser4[rm(25870)]: sibling_list_remove (fs/reiser4/tree_walk.c:813)[zam-32245]: Kernel panic - not syncing: reiser4[rm(25870)]: sibling_list_remove (fs/reiser4/tree_walk.c:813)[zam-32245]: http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060720/fsck3_--check.txt.gz http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060720/fsck3_--fix.txt.gz http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060720/fsck3_--check_after_--fix.txt.gz The fourth was another repeat: reiser4 panicked cowardly: reiser4[pdflush(198)]: capture_anonymous_pages (fs/reiser4/plugin/file/file.c:1007)[vs-49]: Kernel panic - not syncing: reiser4[pdflush(198)]: capture_anonymous_pages (fs/reiser4/plugin/file/file.c:1007)[vs-49]: http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060720/fsck4_--check.txt.gz http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060720/fsck4_--fix.txt.gz http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060720/fsck4_--check_after_fix.txt.gz Where the fsck logs from tests 3 and 4 say entries were removed, they mean it. Those files were GONE. I would expect this to happen to temporary files being written during the panic, but header files should only have been open for reading if at all. I have metadata dumps from before and after one of the fsck --fix runs. Should I make them available? On Wed, 2006-07-19 at 18:07 +0400, Vladimir V. Saveliev wrote: Hello On Wed, 2006-07-19 at 07:28 -0600, Jake Maciejewski wrote: Thanks. Now with debug enabled I've gotten: http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060719/panic1.txt.gz the attached patch fixes a problem nikita-2967 reports about. Would you please check whther it helps. http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060719/fsck1_--check.txt.gz http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060719/fsck1_--fix.txt.gz http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060719/fsck1_--check_after_--fix.txt.gz and http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060719/messages2.txt.gz followed by http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060719/messages2b.txt.gz and without debug: http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060719/messages3.txt.gz http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060719/fsck3_--check.txt.gz http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060719/fsck3_--fix.txt.gz http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060719/fsck3_--check_after_--fix.txt.gz On Tue, 2006-07-18 at 18:18 +0400, Vladimir V. Saveliev wrote: Hello On Tue, 2006-07-18 at 00:52 -0600, Jake Maciejewski wrote: Thanks for the patch, but I can still reproduce the problem. I've been running the attached program to try to speed up the testing process a bit. Interrupting and restarting the compilation loop also seems to help. ok If I had hours to wait, it would probably crash eventually without additional encouragement, but I'm doing everything as an unprivileged user, so I don't
Re: reiser4: possible circular locking dependency
Hello
On Sun, 2006-07-16 at 12:24 +0200, Laurent Riffard wrote:
Hello,
I just got this warning:
===
[ INFO: possible circular locking dependency detected ]
---
nautilus/3229 is trying to acquire lock:
(txnh-hlock){--..}, at: [e0c8e09b] txn_end+0x191/0x368 [reiser4]
but task is already holding lock:
(atom-alock){--..}, at: [e0c8a640] txnh_get_atom+0xf6/0x39e [reiser4]
which lock already depends on the new lock.
lock validator is too severe. I guess that it thinks that
txnh_get_atom() breaks lock ordering. This warning can be ignored.
the existing dependency chain (in reverse order) is:
- #1 (atom-alock){--..}:
[c012d1a7] lock_acquire+0x60/0x80
[c02924f8] _spin_lock+0x19/0x28
[e0c8bbd7] try_capture+0x7cf/0x1cd7 [reiser4]
[e0c786e1] longterm_lock_znode+0x427/0x84f [reiser4]
[e0ca7fcb] seal_validate+0x221/0x5ee [reiser4]
[e0cbefb1] update_sd+0x1c5/0x6a0 [reiser4]
[e0cbfa37] write_sd_by_inode_common+0x5ab/0x61a [reiser4]
[e0cb428b] reiser4_update_sd+0x83/0x8c [reiser4]
[e0caba2f] reiser4_dirty_inode+0xea/0x143 [reiser4]
[c017062d] __mark_inode_dirty+0x2a/0x156
[c0169650] touch_atime+0x89/0x91
[e0cbe010] readdir_common+0x7da/0x848 [reiser4]
[c0162c4f] vfs_readdir+0x4e/0x7a
[c0162cd9] sys_getdents64+0x5e/0xa0
[c0102c2d] sysenter_past_esp+0x56/0x8d
- #0 (txnh-hlock){--..}:
[c012d1a7] lock_acquire+0x60/0x80
[c02924f8] _spin_lock+0x19/0x28
[e0c8e09b] txn_end+0x191/0x368 [reiser4]
[e0c7f97d] reiser4_exit_context+0x1c2/0x571 [reiser4]
[e0cbe02e] readdir_common+0x7f8/0x848 [reiser4]
[c0162c4f] vfs_readdir+0x4e/0x7a
[c0162cd9] sys_getdents64+0x5e/0xa0
[c0102c2d] sysenter_past_esp+0x56/0x8d
other info that might help us debug this:
2 locks held by nautilus/3229:
#0: (inode-i_mutex){--..}, at: [c02913aa] mutex_lock+0x21/0x24
#1: (atom-alock){--..}, at: [e0c8a640] txnh_get_atom+0xf6/0x39e
[reiser4]
stack backtrace:
[c0104db5] show_trace+0xd/0x10
[c0104dd1] dump_stack+0x19/0x1c
[c012bfda] print_circular_bug_tail+0x59/0x64
[c012cfb6] __lock_acquire+0x814/0x9a5
[c012d1a7] lock_acquire+0x60/0x80
[c02924f8] _spin_lock+0x19/0x28
[e0c8e09b] txn_end+0x191/0x368 [reiser4]
[e0c7f97d] reiser4_exit_context+0x1c2/0x571 [reiser4]
[e0cbe02e] readdir_common+0x7f8/0x848 [reiser4]
[c0162c4f] vfs_readdir+0x4e/0x7a
[c0162cd9] sys_getdents64+0x5e/0xa0
[c0102c2d] sysenter_past_esp+0x56/0x8d
I'm running 2.6.18-rc1-mm2 with a few patches:
- drivers-base-check-errors-fix.patch (hot-fixes)
- annotate-pktcdvd-natural-device-hierarchy.patch (from Arjan van de Ven)
- blkdev_get_partition.patch (from Peter Osterlund)
.config is attached
--
laurent
plain text document attachment (config-2.6.18-rc1-mm2)
#
# Automatically generated make config: don't edit
# Linux kernel version: 2.6.18-rc1-mm2
# Sun Jul 16 12:05:34 2006
#
CONFIG_X86_32=y
CONFIG_GENERIC_TIME=y
CONFIG_LOCKDEP_SUPPORT=y
CONFIG_STACKTRACE_SUPPORT=y
CONFIG_SEMAPHORE_SLEEPERS=y
CONFIG_X86=y
CONFIG_MMU=y
CONFIG_GENERIC_ISA_DMA=y
CONFIG_GENERIC_IOMAP=y
CONFIG_GENERIC_HWEIGHT=y
CONFIG_ARCH_MAY_HAVE_PC_FDC=y
CONFIG_DMI=y
CONFIG_DEFCONFIG_LIST=/lib/modules/$UNAME_RELEASE/.config
#
# Code maturity level options
#
CONFIG_EXPERIMENTAL=y
CONFIG_BROKEN_ON_SMP=y
CONFIG_INIT_ENV_ARG_LIMIT=32
#
# General setup
#
CONFIG_LOCALVERSION=
CONFIG_LOCALVERSION_AUTO=y
CONFIG_SWAP=y
CONFIG_SWAP_PREFETCH=y
CONFIG_SYSVIPC=y
# CONFIG_IPC_NS is not set
CONFIG_POSIX_MQUEUE=y
# CONFIG_BSD_PROCESS_ACCT is not set
# CONFIG_TASKSTATS is not set
CONFIG_SYSCTL=y
CONFIG_SYSCTL_SYSCALL=y
# CONFIG_UTS_NS is not set
# CONFIG_AUDIT is not set
# CONFIG_IKCONFIG is not set
# CONFIG_RELAY is not set
CONFIG_INITRAMFS_SOURCE=
CONFIG_KLIBC_ERRLIST=y
CONFIG_KLIBC_ZLIB=y
CONFIG_UID16=y
CONFIG_CC_OPTIMIZE_FOR_SIZE=y
CONFIG_EMBEDDED=y
CONFIG_KALLSYMS=y
CONFIG_KALLSYMS_ALL=y
# CONFIG_KALLSYMS_EXTRA_PASS is not set
CONFIG_HOTPLUG=y
CONFIG_PRINTK=y
CONFIG_BUG=y
CONFIG_ELF_CORE=y
CONFIG_BASE_FULL=y
CONFIG_RT_MUTEXES=y
CONFIG_FUTEX=y
CONFIG_EPOLL=y
CONFIG_SHMEM=y
CONFIG_SLAB=y
CONFIG_VM_EVENT_COUNTERS=y
# CONFIG_TINY_SHMEM is not set
CONFIG_BASE_SMALL=0
# CONFIG_SLOB is not set
#
# Loadable module support
#
CONFIG_MODULES=y
CONFIG_MODULE_UNLOAD=y
CONFIG_MODULE_FORCE_UNLOAD=y
# CONFIG_MODVERSIONS is not set
# CONFIG_MODULE_SRCVERSION_ALL is not set
CONFIG_KMOD=y
#
# Block layer
#
# CONFIG_LBD is not set
# CONFIG_BLK_DEV_IO_TRACE is not set
# CONFIG_LSF is not set
#
# IO Schedulers
#
CONFIG_IOSCHED_NOOP=y
CONFIG_IOSCHED_AS=y
CONFIG_IOSCHED_DEADLINE=y
CONFIG_IOSCHED_CFQ=y
# CONFIG_DEFAULT_AS is not set
# CONFIG_DEFAULT_DEADLINE is not set
CONFIG_DEFAULT_CFQ=y
# CONFIG_DEFAULT_NOOP is
Re: reiser4 OOPSes and panics when out of memory
Hello On Tue, 2006-07-18 at 00:52 -0600, Jake Maciejewski wrote: Thanks for the patch, but I can still reproduce the problem. I've been running the attached program to try to speed up the testing process a bit. Interrupting and restarting the compilation loop also seems to help. ok If I had hours to wait, it would probably crash eventually without additional encouragement, but I'm doing everything as an unprivileged user, so I don't think my tests are unreasonable. Anyway, I'm still getting a panic with debug enabled: reiser4 panicked cowardly: reiser4[find(16411)]: reiser4_dirty_inode (fs/reiser4/super_ops.c:173)[]: Kernel panic - not syncing: reiser4[find(16411)]: reiser4_dirty_inode (fs/reiser4/super_ops.c:173)[]: The attached patch should fix the above. Without debug enabled I've seen: http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060718/messages1.txt.gz http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060718/fsck1_--check.txt.gz http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060718/fsck1_--fix.txt.gz http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060718/fsck1_--check_after_--fix.txt.gz but usually I get: http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060718/messages3.txt.gz with no corruption (although I've been rebooting before complete failure). On Mon, 2006-07-17 at 21:38 +0400, Vladimir V. Saveliev wrote: Hello On Mon, 2006-07-17 at 18:10 +0400, Vladimir V. Saveliev wrote: Hello On Sun, 2006-07-16 at 12:44 -0500, [EMAIL PROTECTED] wrote: Has my previous post (http://marc.theaimsgroup.com/?l=reiserfsm=115259665831650w=2) been overlooked, or have I not provided enough information? Do I need to reproduce these issues on 2.6.18-rc1-mm2? Should I be trying any patches? please try the attached patch. your test crashes reiser4 on my test box. I hope to get a patch ready later today. Not sure that I got the same problem as you, though. We will see. The bottom line is with 2.6.17-mm6, I've always been able to OOPs or panic reiser4 on my amd64 machine (haven't tried x86 yet) by using all available physical memory. readdir has to txn_restart, therefore, grab space for stat data update has to be forced --- commit f1cea9c0a8db0977077d54562677514d6d736690 tree 95479706299276d7c23efd337216501b0dfd69c2 parent bbf99f71c140d7758a6223a557fa4a4d2b5c384e author Vladimir V. Saveliev [EMAIL PROTECTED] Tue, 18 Jul 2006 18:13:05 +0400 committer Vladimir V. Saveliev [EMAIL PROTECTED] Tue, 18 Jul 2006 18:13:05 +0400 plugin/file_ops_readdir.c |2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/plugin/file_ops_readdir.c b/plugin/file_ops_readdir.c index 04ada21..dfdb68d 100644 --- a/plugin/file_ops_readdir.c +++ b/plugin/file_ops_readdir.c @@ -629,7 +629,7 @@ int readdir_common(struct file *f /* dir detach_fsdata(f); /* try to update directory's atime */ - if (reiser4_grab_space(inode_file_plugin(inode)-estimate.update(inode), + if (reiser4_grab_space_force(inode_file_plugin(inode)-estimate.update(inode), BA_CAN_COMMIT) != 0) warning(, failed to update atime on readdir: %llu, get_inode_oid(inode));
Re: reiser4 OOPSes and panics when out of memory
Hello On Sun, 2006-07-16 at 12:44 -0500, [EMAIL PROTECTED] wrote: Has my previous post (http://marc.theaimsgroup.com/?l=reiserfsm=115259665831650w=2) been overlooked, or have I not provided enough information? Do I need to reproduce these issues on 2.6.18-rc1-mm2? Should I be trying any patches? your test crashes reiser4 on my test box. I hope to get a patch ready later today. Not sure that I got the same problem as you, though. We will see. The bottom line is with 2.6.17-mm6, I've always been able to OOPs or panic reiser4 on my amd64 machine (haven't tried x86 yet) by using all available physical memory.
Re: reiser4 OOPSes and panics when out of memory
Hello
On Mon, 2006-07-17 at 18:10 +0400, Vladimir V. Saveliev wrote:
Hello
On Sun, 2006-07-16 at 12:44 -0500, [EMAIL PROTECTED] wrote:
Has my previous post
(http://marc.theaimsgroup.com/?l=reiserfsm=115259665831650w=2) been
overlooked, or have I not provided enough information? Do I need to
reproduce these issues on 2.6.18-rc1-mm2? Should I be trying any patches?
please try the attached patch.
your test crashes reiser4 on my test box. I hope to get a patch ready
later today. Not sure that I got the same problem as you, though. We
will see.
The bottom line is with 2.6.17-mm6, I've always been able to OOPs or panic
reiser4 on my amd64 machine (haven't tried x86 yet) by using all available
physical memory.
diff -puN fs/reiser4/plugin/file_ops_readdir.c~reiser4-fix-readdir fs/reiser4/plugin/file_ops_readdir.c
--- linux-2.6.17-mm6/fs/reiser4/plugin/file_ops_readdir.c~reiser4-fix-readdir 2006-07-17 18:43:50.0 +0400
+++ linux-2.6.17-mm6-vs/fs/reiser4/plugin/file_ops_readdir.c 2006-07-17 19:02:38.0 +0400
@@ -349,6 +349,7 @@ feed_entry(struct file *f,
*/
assert(nikita-3436, lock_stack_isclean(get_current_lock_stack()));
+ txn_restart_current();
result = filldir(dirent, name, (int)strlen(name),
/* offset of this entry */
f-f_pos,
diff -puN fs/reiser4/plugin/file/file.c~reiser4-fix-readdir fs/reiser4/plugin/file/file.c
--- linux-2.6.17-mm6/fs/reiser4/plugin/file/file.c~reiser4-fix-readdir 2006-07-17 20:39:11.0 +0400
+++ linux-2.6.17-mm6-vs/fs/reiser4/plugin/file/file.c 2006-07-17 21:35:31.0 +0400
@@ -781,9 +781,12 @@ int find_or_create_extent(struct page *p
lock_page(page);
node = jnode_of_page(page);
- unlock_page(page);
- if (IS_ERR(node))
+ if (IS_ERR(node)) {
+ unlock_page(page);
return PTR_ERR(node);
+ }
+ JF_SET(node, JNODE_WRITE_PREPARED);
+ unlock_page(page);
if (node-blocknr == 0) {
plugged_hole = 0;
@@ -791,6 +794,7 @@ int find_or_create_extent(struct page *p
(loff_t)page-index PAGE_CACHE_SHIFT,
plugged_hole);
if (result) {
+ JF_CLR(node, JNODE_WRITE_PREPARED);
jput(node);
warning(, update_extent failed: %d, result);
return result;
@@ -806,6 +810,7 @@ int find_or_create_extent(struct page *p
}
BUG_ON(node-atom == NULL);
+ JF_CLR(node, JNODE_WRITE_PREPARED);
jput(node);
if (get_current_context()-entd) {
@@ -1729,7 +1734,9 @@ ssize_t read_unix_file(struct file *file
return RETERR(-EFAULT);
}
- read = read_file(hint, file, buf, left, off);
+ read = read_file(hint, file, buf,
+ left PAGE_CACHE_SIZE ? PAGE_CACHE_SIZE : left,
+ off);
drop_nonexclusive_access(uf_info);
_
Re: reiser4 OOPSes and panics when out of memory
Thanks for the patch, but I can still reproduce the problem. I've been
running the attached program to try to speed up the testing process a
bit. Interrupting and restarting the compilation loop also seems to
help.
If I had hours to wait, it would probably crash eventually without
additional encouragement, but I'm doing everything as an unprivileged
user, so I don't think my tests are unreasonable.
Anyway, I'm still getting a panic with debug enabled:
reiser4 panicked cowardly: reiser4[find(16411)]: reiser4_dirty_inode
(fs/reiser4/super_ops.c:173)[]:
Kernel panic - not syncing: reiser4[find(16411)]: reiser4_dirty_inode
(fs/reiser4/super_ops.c:173)[]:
Without debug enabled I've seen:
http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060718/messages1.txt.gz
http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060718/fsck1_--check.txt.gz
http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060718/fsck1_--fix.txt.gz
http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060718/fsck1_--check_after_--fix.txt.gz
but usually I get:
http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060718/messages3.txt.gz
with no corruption (although I've been rebooting before complete
failure).
On Mon, 2006-07-17 at 21:38 +0400, Vladimir V. Saveliev wrote:
Hello
On Mon, 2006-07-17 at 18:10 +0400, Vladimir V. Saveliev wrote:
Hello
On Sun, 2006-07-16 at 12:44 -0500, [EMAIL PROTECTED] wrote:
Has my previous post
(http://marc.theaimsgroup.com/?l=reiserfsm=115259665831650w=2) been
overlooked, or have I not provided enough information? Do I need to
reproduce these issues on 2.6.18-rc1-mm2? Should I be trying any patches?
please try the attached patch.
your test crashes reiser4 on my test box. I hope to get a patch ready
later today. Not sure that I got the same problem as you, though. We
will see.
The bottom line is with 2.6.17-mm6, I've always been able to OOPs or panic
reiser4 on my amd64 machine (haven't tried x86 yet) by using all available
physical memory.
--
Jake Maciejewski [EMAIL PROTECTED]
#include sys/sysinfo.h
#include unistd.h
#include stdlib.h
#include stdio.h
#include errno.h
#include string.h
#define MB 1048576
#define DEFAULT_SAVE 0
#define DELAY 10
int main(int argc, char *argv[]) {
struct sysinfo info;
size_t waste, save;
void *ptr;
sysinfo(info);
if(argc1)
save = atoi(argv[1])*MB;
else
save = DEFAULT_SAVE*MB;
if( save = info.freeram ) {
fprintf(stderr, error: only %iMB free\n, (int)info.freeram/MB);
return ENOMEM;
}
while(1) {
sysinfo(info);
waste = (info.freeram-save)0 ? info.freeram-save : 0;
ptr = malloc(waste);
if( ptr == 0 ) {
perror(malloc() failed\n);
return ENOMEM;
}
printf(reserving %iMB, wasting %iMB\n, (int)save/MB, (int)waste/MB);
memset(ptr, 0, waste);
sleep(DELAY);
free(ptr);
}
return 0;
}
Re: reiser4 oops
I tried 2.6.17-mm6 (was the latest at the time) and reiser4 still always fails when it runs out of memory. My test has been building a kernel with enough jobs to exhaust memory. Expecting problems, I began by testing reiser4 with debugging enabled. The first time, I got the panic below after running while true ; do make mrproper zcat /proc/config.gz .config make -j64 ; done in linux-2.6.17-mm6 (on a fresh test filesystem) for an unknown duration of time. reiser4 panicked cowardly: reiser4[cc1(12238)]: page_bio (fs/reiser4/page_cache.c:434)[nikita-2276]: Kernel panic - not syncing: reiser4[cc1(12238)]: page_bio (fs/reiser4/page_cache.c:434)[nikita-2276]: The next two times, after about half an hour of the above command I got the panic below (except the second time was with cc1 instead of fixdep). reiser4 panicked cowardly: reiser4[fixdep(19463)]: sibling_list_remove (fs/reiser4/tree_walk.c:813)[zam-32245]: Kernel panic - not syncing: reiser4[fixdep(19463)]: sibling_list_remove (fs/reiser4/tree_walk.c:813)[zam-32245]: The third time the filesystem wasn't clean after a reboot. fsck.reiser4 --check logged: http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060711/fsck3_--check.txt.gz fsck.reiser4 --fix logged: http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060711/fsck3_--fix.txt.gz Another --check turned up clean: http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060711/fsck3_--check_after_--fix.txt.gz Things got slightly more interesting with debugging disabled. The first time I got the following OOPS after some flushing like mad messages. A bunch of cc1 process were stuck in the D state. http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060711/messages4.txt.gz http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060711/fsck4_--check.txt.gz http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060711/fsck4_--fix.txt.gz http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060711/fsck4_--check_after_--fix.txt.gz The second time I got: http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060711/messages5.txt.gz http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060711/fsck5_--check.txt.gz http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060711/fsck5_--fix.txt.gz http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060711/fsck5_--check_after_--fix.txt.gz On Fri, 2006-07-07 at 17:16 +0400, Vladimir V. Saveliev wrote: Hello On Fri, 2006-07-07 at 01:18 -0600, Jake Maciejewski wrote: It doesn't look like this issue has been fixed: http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060702/reiser4_kio_http.txt.gz Again fsck showed a clean filesystem. Should I try -mm? Yes, please try to check whether latest mm has this problem. This is somewhat difficult to reproduce because the critical factor seems to be using all available memory, however in doing so the system quickly becomes unresponsive. On Mon, 2006-07-03 at 17:23 -0600, Jake Maciejewski wrote: Thanks for the patch. I don't think I'll be able to test it before Wednesday, though. On Mon, 2006-07-03 at 14:10 +0400, Vladimir V. Saveliev wrote: Hello On Sun, 2006-07-02 at 22:24 -0600, Jake Maciejewski wrote: I'm seeing this on 2.6.16.20 with the -4 patch, amd64 with preempt. The OOM killer was called even though I have 1GB RAM and 4GB swap. My logs are available at: http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060702/oom.txt.gz Both affected filesystems (rsync was using one filesystem, cc1 the other) came up clean with fsck. On Sat, 2006-07-01 at 22:08 +0200, Łukasz Mierzwa wrote: I'm running x86 gentoo system with / on reiser4, I'm using suspend-sources-2.6.16-r8 kernel with 2.6.16-4 patch. Today I run emerge --sync, after that I started to compile new xine-lib while browsing net, when xine-lib was finishing I got this errors: kernel BUG at fs/inode.c:253! would you please check whether this patch helps? ftp://ftp.ru.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.17/2.6.17-mm5/broken-out/reiser4-run-truncate_inode_pages-in-reiser4_delete_inode.patch invalid opcode: [#1] PREEMPT Modules linked in: ndiswrapper fglrx acer_acpi nsc_ircc irda crc_ccitt snd_atiixp snd_ac97_codec snd_ac97_bus yenta_socket rsrc_nonstatic pcmcia_core tifm_7xx1 tifm_core CPU:0 EIP:0060:[c017bc56]Tainted: P VLI EFLAGS: 00010206 (2.6.16-suspend2-r8 #2) EIP is at clear_inode+0x16/0xb0 eax: 0003 ebx: d5245d20 ecx: edx: d5245df8 esi: df2099c0 edi: c2a3a000 ebp: c2a3a000 esp: c2a3be94 ds: 007b es: 007b ss: 0068 Process emerge (pid: 9123,
Re: reiser4 oops
It doesn't look like this issue has been fixed: http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060702/reiser4_kio_http.txt.gz Again fsck showed a clean filesystem. Should I try -mm? This is somewhat difficult to reproduce because the critical factor seems to be using all available memory, however in doing so the system quickly becomes unresponsive. On Mon, 2006-07-03 at 17:23 -0600, Jake Maciejewski wrote: Thanks for the patch. I don't think I'll be able to test it before Wednesday, though. On Mon, 2006-07-03 at 14:10 +0400, Vladimir V. Saveliev wrote: Hello On Sun, 2006-07-02 at 22:24 -0600, Jake Maciejewski wrote: I'm seeing this on 2.6.16.20 with the -4 patch, amd64 with preempt. The OOM killer was called even though I have 1GB RAM and 4GB swap. My logs are available at: http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060702/oom.txt.gz Both affected filesystems (rsync was using one filesystem, cc1 the other) came up clean with fsck. On Sat, 2006-07-01 at 22:08 +0200, Łukasz Mierzwa wrote: I'm running x86 gentoo system with / on reiser4, I'm using suspend-sources-2.6.16-r8 kernel with 2.6.16-4 patch. Today I run emerge --sync, after that I started to compile new xine-lib while browsing net, when xine-lib was finishing I got this errors: kernel BUG at fs/inode.c:253! would you please check whether this patch helps? ftp://ftp.ru.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.17/2.6.17-mm5/broken-out/reiser4-run-truncate_inode_pages-in-reiser4_delete_inode.patch invalid opcode: [#1] PREEMPT Modules linked in: ndiswrapper fglrx acer_acpi nsc_ircc irda crc_ccitt snd_atiixp snd_ac97_codec snd_ac97_bus yenta_socket rsrc_nonstatic pcmcia_core tifm_7xx1 tifm_core CPU:0 EIP:0060:[c017bc56]Tainted: P VLI EFLAGS: 00010206 (2.6.16-suspend2-r8 #2) EIP is at clear_inode+0x16/0xb0 eax: 0003 ebx: d5245d20 ecx: edx: d5245df8 esi: df2099c0 edi: c2a3a000 ebp: c2a3a000 esp: c2a3be94 ds: 007b es: 007b ss: 0068 Process emerge (pid: 9123, threadinfo=c2a3a000 task=c2a0ba70) Stack: 0df2099c0 d5245d20 df2099c0 c01db519 c017a385 0400 d5245e28 d5245d20 c2a3a000 d415c114 d5245d20 c01db4e0 c017bd59 d5245d20 dfe9b000 c017b81c d415c114 c017a431 dfe9b000 df533000 c2a3bf50 c0172d74 Call Trace: [c01db519] reiser4_delete_inode+0x39/0xc0 [c017a385] dput+0x25/0x170 [c01db4e0] reiser4_delete_inode+0x0/0xc0 [c017bd59] generic_delete_inode+0x69/0x100 [c017b81c] iput+0x5c/0x70 [c017a431] dput+0xd1/0x170 [c0172d74] sys_renameat+0x1c4/0x1f0 [c0172dc7] sys_rename+0x27/0x30 [c0102ef3] sysenter_past_esp+0x54/0x75 Code: 80 98 00 00 00 8b 40 38 eb c4 8d 74 26 00 8d bc 27 00 00 00 00 56 53 89 c3 83 ec 04 e8 14 b0 fe ff 8b 83 c4 00 00 00 85 c0 74 08 0f 0b fd 00 ab d4 37 c0 8b 83 20 01 00 00 a8 10 75 08 0f 0b ff 4reiser4[emerge(9123)]: release_unix_file (fs/reiser4/plugin/file/file.c:2294)[vs-44]: WARNING: out of memory? reiser4[emerge(9123)]: release_unix_file (fs/reiser4/plugin/file/file.c:2294)[vs-44]: WARNING: out of memory? reiser4[emerge(9123)]: release_unix_file (fs/reiser4/plugin/file/file.c:2294)[vs-44]: WARNING: out of memory? reiser4[emerge(9123)]: release_unix_file (fs/reiser4/plugin/file/file.c:2294)[vs-44]: WARNING: out of memory? reiser4[emerge(9123)]: release_unix_file (fs/reiser4/plugin/file/file.c:2294)[vs-44]: WARNING: out of memory? reiser4[emerge(9123)]: release_unix_file (fs/reiser4/plugin/file/file.c:2294)[vs-44]: WARNING: out of memory? reiser4[emerge(9123)]: release_unix_file (fs/reiser4/plugin/file/file.c:2294)[vs-44]: WARNING: out of memory? reiser4[emerge(9123)]: release_unix_file (fs/reiser4/plugin/file/file.c:2294)[vs-44]: WARNING: out of memory? reiser4[emerge(9123)]: release_unix_file (fs/reiser4/plugin/file/file.c:2294)[vs-44]: WARNING: out of memory? reiser4[emerge(9123)]: release_unix_file (fs/reiser4/plugin/file/file.c:2294)[vs-44]:
Re: reiser4 oops
Hello On Fri, 2006-07-07 at 01:18 -0600, Jake Maciejewski wrote: It doesn't look like this issue has been fixed: http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060702/reiser4_kio_http.txt.gz Again fsck showed a clean filesystem. Should I try -mm? Yes, please try to check whether latest mm has this problem. This is somewhat difficult to reproduce because the critical factor seems to be using all available memory, however in doing so the system quickly becomes unresponsive. On Mon, 2006-07-03 at 17:23 -0600, Jake Maciejewski wrote: Thanks for the patch. I don't think I'll be able to test it before Wednesday, though. On Mon, 2006-07-03 at 14:10 +0400, Vladimir V. Saveliev wrote: Hello On Sun, 2006-07-02 at 22:24 -0600, Jake Maciejewski wrote: I'm seeing this on 2.6.16.20 with the -4 patch, amd64 with preempt. The OOM killer was called even though I have 1GB RAM and 4GB swap. My logs are available at: http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060702/oom.txt.gz Both affected filesystems (rsync was using one filesystem, cc1 the other) came up clean with fsck. On Sat, 2006-07-01 at 22:08 +0200, Łukasz Mierzwa wrote: I'm running x86 gentoo system with / on reiser4, I'm using suspend-sources-2.6.16-r8 kernel with 2.6.16-4 patch. Today I run emerge --sync, after that I started to compile new xine-lib while browsing net, when xine-lib was finishing I got this errors: kernel BUG at fs/inode.c:253! would you please check whether this patch helps? ftp://ftp.ru.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.17/2.6.17-mm5/broken-out/reiser4-run-truncate_inode_pages-in-reiser4_delete_inode.patch invalid opcode: [#1] PREEMPT Modules linked in: ndiswrapper fglrx acer_acpi nsc_ircc irda crc_ccitt snd_atiixp snd_ac97_codec snd_ac97_bus yenta_socket rsrc_nonstatic pcmcia_core tifm_7xx1 tifm_core CPU:0 EIP:0060:[c017bc56]Tainted: P VLI EFLAGS: 00010206 (2.6.16-suspend2-r8 #2) EIP is at clear_inode+0x16/0xb0 eax: 0003 ebx: d5245d20 ecx: edx: d5245df8 esi: df2099c0 edi: c2a3a000 ebp: c2a3a000 esp: c2a3be94 ds: 007b es: 007b ss: 0068 Process emerge (pid: 9123, threadinfo=c2a3a000 task=c2a0ba70) Stack: 0df2099c0 d5245d20 df2099c0 c01db519 c017a385 0400 d5245e28 d5245d20 c2a3a000 d415c114 d5245d20 c01db4e0 c017bd59 d5245d20 dfe9b000 c017b81c d415c114 c017a431 dfe9b000 df533000 c2a3bf50 c0172d74 Call Trace: [c01db519] reiser4_delete_inode+0x39/0xc0 [c017a385] dput+0x25/0x170 [c01db4e0] reiser4_delete_inode+0x0/0xc0 [c017bd59] generic_delete_inode+0x69/0x100 [c017b81c] iput+0x5c/0x70 [c017a431] dput+0xd1/0x170 [c0172d74] sys_renameat+0x1c4/0x1f0 [c0172dc7] sys_rename+0x27/0x30 [c0102ef3] sysenter_past_esp+0x54/0x75 Code: 80 98 00 00 00 8b 40 38 eb c4 8d 74 26 00 8d bc 27 00 00 00 00 56 53 89 c3 83 ec 04 e8 14 b0 fe ff 8b 83 c4 00 00 00 85 c0 74 08 0f 0b fd 00 ab d4 37 c0 8b 83 20 01 00 00 a8 10 75 08 0f 0b ff 4reiser4[emerge(9123)]: release_unix_file (fs/reiser4/plugin/file/file.c:2294)[vs-44]: WARNING: out of memory? reiser4[emerge(9123)]: release_unix_file (fs/reiser4/plugin/file/file.c:2294)[vs-44]: WARNING: out of memory? reiser4[emerge(9123)]: release_unix_file (fs/reiser4/plugin/file/file.c:2294)[vs-44]: WARNING: out of memory? reiser4[emerge(9123)]: release_unix_file (fs/reiser4/plugin/file/file.c:2294)[vs-44]: WARNING: out of memory? reiser4[emerge(9123)]: release_unix_file (fs/reiser4/plugin/file/file.c:2294)[vs-44]: WARNING: out of memory? reiser4[emerge(9123)]: release_unix_file (fs/reiser4/plugin/file/file.c:2294)[vs-44]: WARNING: out of memory? reiser4[emerge(9123)]: release_unix_file (fs/reiser4/plugin/file/file.c:2294)[vs-44]: WARNING: out of memory? reiser4[emerge(9123)]: release_unix_file (fs/reiser4/plugin/file/file.c:2294)[vs-44]: WARNING: out of memory?
Re: Reiser4 cowardly panic 2.6.17-mm5
This seems to be fixed with 2.6.17-mm6. -Joe Joe Feise writes: I consistently get cowardly panic errors during bootup with 2.6.17-mm5 (I also had that with -mm3, and had hoped -mm5 would have fixed it.) It corrupts the partition this is happening on (/var) I don't have dmesg or syslog output, though, since this happens during bootup, and /var gets corrupted... -Joe
Re: Reiser4 on top of software raid 0 (dmraid) - another problem
Hello On Tue, 2006-07-04 at 14:56 -0400, Diego Pinheiro wrote: I'm having another problem when running reiser4 on top of software raid 0 (dmraid)... I can work the filesystem fine (mount, copy, create, delete, move, etc...) - even big ammount of data ( 3Gb) as a separate (non / - root folder) filesystem. But if I try to mount this filesystem as the root filesystem I get a segmentation fault in the init script (in the process to update the /etc/mtab file (line 413 Fedora Core 5 init script). The line mount -f / goes fine; but the line mount -f /proc generates the error)... We need to see the error. Would you, please, setup serial or net console and catch the output? The same procedure with a ext3 filesystem goes fine, so apparently it's not the init script, neither the software raid or the kernel (2.6.16.22 with 2.6.16-4 reiser4 patch) Any ideas? Thanks, Diego
Re: Reiser4 Kernel OOPS on fs/reiser4/plugin/file/file.c:2294 (perhaps the same as the one reported in 2006-07-03 22:24:29?)
Hello On Tue, 2006-07-04 at 17:56 -0500, Skotlex wrote: Hello. Am new to this mailing list.. and I only joined because I can't find any other place to look up bug reports and the like. Anyway, I have a kernel oops on Kernel 2.6.16 with the patchset -4. Is this the same as the one from that other mail posted on the mailing archives, which I have no idea how to reference it from here... the meta-data of such email is: List: reiserfs Subject:Re: reiser4 oops From: Jake Maciejewski maciejej () msoe ! edu Date: 2006-07-03 22:24:29 Message-ID: 1151969007.10172.23.camel () gentoo Looking at the trace, it hang on the same file, same line. Should I try the patch provided on that email, then? Point worth noting: The problem no longer is present when running 2.6.14, -1 patch Oops: [#1] Modules linked in: sch_ingress cls_u32 usblp cdc_ether joydev nvidia usbnet evdev ehci_hcd ohci_hcd CPU:0 EIP:0060:[c01310f5]Tainted: P VLI EFLAGS: 00010213 (2.6.16-reiser4-r9 #2) EIP is at truncate_inode_pages_range+0x38/0x275 eax: ebx: 1fff ecx: 1fff edx: esi: edi: 0001 ebp: esp: eb737860 ds: 007b es: 007b ss: 0068 Process ldconfig (pid: 5748, threadinfo=eb736000 task=f66e4050) Stack: 0 eb7378a4 0001 eb7378a4 eb7378a4 c1717f80 c0130df6 eb7378ac 0001 0040 0002 0010 003f eb356c3c c01de9b1 eb356c3c c1717080 0002 0001 c01a0f57 Call Trace: [c0130df6] __pagevec_release+0x18/0x23 [c01de9b1] radix_tree_gang_lookup+0x3c/0x54 [c01a0f57] invalidate_unformatted+0x41/0x65 [c01a100f] truncate_jnodes_range+0x94/0xcd [c01be47b] kill_hook_extent+0x3f0/0x557 [c01be6e5] kill_units_extent+0x103/0x230 [c01b5d35] kill_tail+0x0/0x32 [c01b5ce2] kill_units+0x0/0x53 [c01b55ed] parse_cut+0x3e/0x694 [c01b5d67] kill_head+0x0/0x24 [c01b5d2b] kill_units+0x49/0x53 [c01b5ce2] kill_units+0x0/0x53 [c01b5d84] kill_head+0x1d/0x24 [c01b6022] prepare_for_compact+0x1ea/0x411 [c01905b3] jload_gfp+0xf3/0x105 [c01b626c] kill_node40+0x23/0x9a [c0192766] lock_carry_node_tail+0x16/0x18 [c0193f16] carry_cut+0x3f/0x4c [c019214e] carry_on_level+0x30/0xa0 [c019204a] carry+0x56/0x12a [c0196273] kill_node_content+0x123/0x13b [c019673c] cut_tree_worker_common+0x19e/0x2fe [c019659e] cut_tree_worker_common+0x0/0x2fe [c019694c] cut_tree_object+0xb0/0x14d [c0196a10] cut_tree+0x27/0x37 [c01ae0a2] extent2tail+0x1d4/0x3c0 [c014d75a] link_path_walk+0xa5/0xaf [c01aad85] find_file_item_nohint+0x2e/0x32 [c01905b3] jload_gfp+0xf3/0x105 [c0194a5f] longterm_unlock_znode+0xac/0xde [c01ac7bc] find_first_item+0x99/0xa4 [c01ac893] open_unix_file+0xcc/0xed [c01ac7c7] open_unix_file+0x0/0xed [c0141991] __dentry_open+0xb4/0x180 [c0141b33] nameidata_to_filp+0x1f/0x31 [c0141a91] do_filp_open+0x34/0x3c [c0141bd8] get_unused_fd+0x4c/0x91 [c0141cbd] do_sys_open+0x40/0xb6 [c0141d46] sys_open+0x13/0x17 [c0102397] sysenter_past_esp+0x54/0x75 Code: 24 68 8b 5c 24 6c 8b 74 24 70 89 c7 89 d5 81 c7 ff 0f 00 00 83 d5 00 25 ff 0f 00 00 89 04 24 8b 44 24 60 0f ac ef 0c 89 7c 24 08 83 78 28 00 0f 84 2b 02 00 00 89 d8 31 d2 25 ff 0f 00 00 89 c1 4reiser4[ldconfig(5748)]: release_unix_file (fs/reiser4/plugin/file/file.c:2294)[vs-44]: WARNING: out of memory? It may be related, or it may not, but here goes the history of how I got it to corrupt like that: - I recently updated to Xorg-X11 7.0 - While trying to get my wacom tablet working again, I noticed that metalog 0.8-rc1 locked up (this makes all processes that try to access the logger lock up badly on a kernel call, so I can't kill them. Gotta report this one to the metalog bugtracker) - On midst of that, I somehow accidentally killed X. But I couldn't get back to the console. - After failing to reboot the machine cleanly connecting through ssh, I tried the alt+printscreen + s - u - b method. - After reboot, I notice that a certain core lib file got truncated (how could /lib/libutil.so.1 get corrupted? This is a lib file, as many processes that were accessing it, none of them should had been writing to it!) - After hours getting a replacement file for that file and restoring glibc, I notice that the previously mentioned oops happens whenever emerge ends, when updating the ld so cache or something like that. After that, any changes to the filesystem are no longer saved. On reboot, the system is effectively restored to the point before that oops. - I can reproduce the oops whenever emerge ends, or when trying to launch a gtk2 app. The trace in both cases lead to the same file:lineno. can you install 2.6.17-mm6 and try to reproduce the problem? Sorry for the messy email format.. I am not used to mailing lists.. and I really was hoping to find/use a bugtracker instead.
Re: reiser4 oops
Hello On Sun, 2006-07-02 at 22:24 -0600, Jake Maciejewski wrote: I'm seeing this on 2.6.16.20 with the -4 patch, amd64 with preempt. The OOM killer was called even though I have 1GB RAM and 4GB swap. My logs are available at: http://people.msoe.edu/~maciejej/patches/AMD64_reiser4_debug/20060702/oom.txt.gz Both affected filesystems (rsync was using one filesystem, cc1 the other) came up clean with fsck. On Sat, 2006-07-01 at 22:08 +0200, Łukasz Mierzwa wrote: I'm running x86 gentoo system with / on reiser4, I'm using suspend-sources-2.6.16-r8 kernel with 2.6.16-4 patch. Today I run emerge --sync, after that I started to compile new xine-lib while browsing net, when xine-lib was finishing I got this errors: kernel BUG at fs/inode.c:253! would you please check whether this patch helps? ftp://ftp.ru.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.17/2.6.17-mm5/broken-out/reiser4-run-truncate_inode_pages-in-reiser4_delete_inode.patch invalid opcode: [#1] PREEMPT Modules linked in: ndiswrapper fglrx acer_acpi nsc_ircc irda crc_ccitt snd_atiixp snd_ac97_codec snd_ac97_bus yenta_socket rsrc_nonstatic pcmcia_core tifm_7xx1 tifm_core CPU:0 EIP:0060:[c017bc56]Tainted: P VLI EFLAGS: 00010206 (2.6.16-suspend2-r8 #2) EIP is at clear_inode+0x16/0xb0 eax: 0003 ebx: d5245d20 ecx: edx: d5245df8 esi: df2099c0 edi: c2a3a000 ebp: c2a3a000 esp: c2a3be94 ds: 007b es: 007b ss: 0068 Process emerge (pid: 9123, threadinfo=c2a3a000 task=c2a0ba70) Stack: 0df2099c0 d5245d20 df2099c0 c01db519 c017a385 0400 d5245e28 d5245d20 c2a3a000 d415c114 d5245d20 c01db4e0 c017bd59 d5245d20 dfe9b000 c017b81c d415c114 c017a431 dfe9b000 df533000 c2a3bf50 c0172d74 Call Trace: [c01db519] reiser4_delete_inode+0x39/0xc0 [c017a385] dput+0x25/0x170 [c01db4e0] reiser4_delete_inode+0x0/0xc0 [c017bd59] generic_delete_inode+0x69/0x100 [c017b81c] iput+0x5c/0x70 [c017a431] dput+0xd1/0x170 [c0172d74] sys_renameat+0x1c4/0x1f0 [c0172dc7] sys_rename+0x27/0x30 [c0102ef3] sysenter_past_esp+0x54/0x75 Code: 80 98 00 00 00 8b 40 38 eb c4 8d 74 26 00 8d bc 27 00 00 00 00 56 53 89 c3 83 ec 04 e8 14 b0 fe ff 8b 83 c4 00 00 00 85 c0 74 08 0f 0b fd 00 ab d4 37 c0 8b 83 20 01 00 00 a8 10 75 08 0f 0b ff 4reiser4[emerge(9123)]: release_unix_file (fs/reiser4/plugin/file/file.c:2294)[vs-44]: WARNING: out of memory? reiser4[emerge(9123)]: release_unix_file (fs/reiser4/plugin/file/file.c:2294)[vs-44]: WARNING: out of memory? reiser4[emerge(9123)]: release_unix_file (fs/reiser4/plugin/file/file.c:2294)[vs-44]: WARNING: out of memory? reiser4[emerge(9123)]: release_unix_file (fs/reiser4/plugin/file/file.c:2294)[vs-44]: WARNING: out of memory? reiser4[emerge(9123)]: release_unix_file (fs/reiser4/plugin/file/file.c:2294)[vs-44]: WARNING: out of memory? reiser4[emerge(9123)]: release_unix_file (fs/reiser4/plugin/file/file.c:2294)[vs-44]: WARNING: out of memory? reiser4[emerge(9123)]: release_unix_file (fs/reiser4/plugin/file/file.c:2294)[vs-44]: WARNING: out of memory? reiser4[emerge(9123)]: release_unix_file (fs/reiser4/plugin/file/file.c:2294)[vs-44]: WARNING: out of memory? reiser4[emerge(9123)]: release_unix_file (fs/reiser4/plugin/file/file.c:2294)[vs-44]: WARNING: out of memory? reiser4[emerge(9123)]: release_unix_file (fs/reiser4/plugin/file/file.c:2294)[vs-44]: WARNING: out of memory? reiser4[emerge(9123)]: release_unix_file (fs/reiser4/plugin/file/file.c:2294)[vs-44]: WARNING: out of memory? reiser4[emerge(9123)]: release_unix_file (fs/reiser4/plugin/file/file.c:2294)[vs-44]: WARNING: out of memory? reiser4[emerge(9123)]: release_unix_file (fs/reiser4/plugin/file/file.c:2294)[vs-44]: WARNING: out of memory? reiser4[emerge(9123)]: release_unix_file (fs/reiser4/plugin/file/file.c:2294)[vs-44]: WARNING: out of memory?
