Relation between nr_dirty and nr_inactive

2007-11-29 Thread Kunal Trivedi 
Hi,
I am running older kernel (CentOS 2.6.9-34 SMP) on 32 bit arch. Some
of my systems got hung, while trying to write some data to disk. All
those systems exhibit similar pattern where during this time,
/proc/meminfo suggesting 'Inactive' < 'Dirty'. All of machines have 2G
of physical memory and ~1.5G memory is locked (via mlock).

I tried reading code but could not establish any direct relationship
between Zone->in_active pages vs. per-cpu_page_state->nr_dirty.

Has anybody seen system in this kind of state before ? And are these 2
parameters affect each-other ?

Thanks
-Kunal
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Kernel Null pointer dereference in sysfs_readdir()

2007-03-01 Thread Kunal Trivedi 

Hi,
I have experienced kernel OOPS in sysfs_readdir. Below is some
information about kernel version, debugging, some other reports of
OOPS. At the end of the mail I am attaching OOPS.

I also have kernel dump via which I could traverse sysfs_dirent list,
and also see dentry. (Analysis is in #4).

I could not figure out how that happen, just by reading code as most
of the things are under inode lock. But there could be some race which
is causing this crash.

#
1) SMP box with 4GB physical memory.
2) kernel is 2.6.9-34-EL (CentOS)
NOTE: I know its old kernel. But I've compared fs/sysfs code from that
kernel to latest and code had not been changed fundamentally.
Especially OOPS I've got, is very similar to other people have
reported and that portion of code had not been changed.

3) Digging archive, I've found following to threads which reports
similar issue. (Esepcially 2nd link). Though fixes commited for those
problems still not clerify situation I have encountered.
http://lkml.org/lkml/2005/11/23/101
http://lkml.org/lkml/2006/7/12/281

4) Debugging: (Oops message at the end.. And if needed, I can provide
kernel dump files)
EIP:0060:[]
Code arround EIP is
0xe01a40b7 :   movl   %ecx,0x14(%esp,1)
0xe01a40bb :   movl   0x20(%esi),%eax
0xe01a40be :   testl  %eax,%eax
0xe01a40c0 :   je 0xe01a41e8 
0xe01a40c6 :   movl   0x10(%eax),%eax
0xe01a40c9 :   movl   0x18(%eax),%edx <--Crashed Here.

Now, If i traverse sysfs_dirent from 'struct file *', I see following..
struct file *filep = ed6e7280;
And sysfs_dirent are
1) 0xf7ddbbc0
2) 0xf7d6b100
3) 0xf7d6cc88
4) 0xf7d6fe80

1, 2 and 4 all have count = 1.
'esi' = 0xf7d6b100  which is pointer to the sysfs_dirent


p *(struct sysfs_dirent *)0xf7d6b100

struct sysfs_dirent {
   s_count = atomic_t {
   counter = 1
   }
   s_sibling = struct list_head {
   next = 0xf7d6cc8c
   prev = 0xf7ddbbc4
   }
   s_children = struct list_head {
   next = 0xf7d6b10c
   prev = 0xf7d6b10c
   }
   s_element = 0xf76ed280
   s_type = 32
   s_mode = 41471
   s_dentry = (nil)<--
This is null
}

But disass of sysfs_readdir(), shows that this is not possible
otherwise 'testl' would be true and it should have jumped to
sysfs_readdir() + 504 address. But testl succeeded, Not only that, we
dereference that pointer and got d_inode
 Code #
if (next->s_dentry)
  ino = next->s_dentry->d_inode->i_ino

but then while accessing d_inode (to get i_ino), it crashed.

That means, at one point s_dentry was NON-NULL. And next memonet it
became NULL and its inode has gone, causing this OOPS.

5) OOPS messages from console.
   <1>Unable to handle kernel NULL pointer dereference at virtual
address 0018
   <1> printing eip:
   <4>e01a40c9
   <1>*pde = 
   <1>Oops:  [#1]
   <4>SMP
   <4>Modules linked in: ipt_state ip_conntrack iptable_filter
cls_u32 iptable_mangle lm85 i2c_i801 w83627hf_wdt w83627hf i2c_sensor
i2c_isa i2c_core slcmi ip_tables e7xxx_edac edac_mc
   <4>CPU:2
   <4>EIP:0060:[]Tainted: PF VLI
   <4>EFLAGS: 00010286   (2.6.9-34.EL-i386_SMP)
   <4>EIP is at sysfs_readdir+0xd9/0x210
   <4>eax:    ebx: f7d6b104   ecx: 0006   edx: 0020
   <4>esi: f7d6b100   edi: f7f1cb87   ebp: f7f1cb80   esp: ef432f48
   <4>ds: 007b   es: 007b   ss: 0068
   <4>Process sensors (pid: 2933, threadinfo=ef432000 task=f562c030)
   <4>Stack: 0002  016c32f7 000a f7d6cc8c 0006
f7ddbbc4 e017a670
   <4>   ef432fa0 ed6e7280 e0409ba0 ed6e7280 f6f180b0 f6f18120
e017a33f ef432fa0
   <4>   e017a670 09ce61b4 ed6e7280 fff7  e017a81e
09ce6204 09ce61e4
   <4>Call Trace:
   <4> [] filldir64+0x0/0x140
   <4> [] vfs_readdir+0xaf/0xd0
   <4> [] filldir64+0x0/0x140
   <4> [] sys_getdents64+0x6e/0xb6
   <4> [] syscall_call+0x7/0xb
   <4>Code: 26 00 89 f0 e8 89 e8 ff ff 89 c5 b9 ff ff ff ff 31 c0 89
ef f2 ae f7 d1 49 89 4c 24 14 8b 46 20 85 c0 0f 84 22 01 00 00 8b 40
10 <8b> 50 18 0f b7 46 1c 89 54 24 08 8b 4c 24 24 c1 e8 0c 89 44 24

Please advice.

Thanks
-Kunal
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Possible Bug in VM accounting (Committed_AS) on x86_64 architecture ?

2006-12-06 Thread Kunal Trivedi 

Hi Alan,
Sorry for late followup on this.

I did found the problem. It was 32 bit binary running on 64 bit arch.
Actually main kernel had fixed this problem in 2.6.14
(http://www.kernel.org/git/?p=linux/kernel/git/torvalds/linux-2.6.git;a=commit;h=2fd4ef85e0db9ed75c98e13953257a967ea55e03)
But apparently CentOS has not ported it yet.

Thanks for your reply

-Kunal
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: dentry cache grows to really large number on 64bit machine

2006-12-06 Thread Kunal Trivedi 

Hi,
Another piece of information...
If i do on shell.

# while 1
# ./grow_dentry
# end
(grow_dentry is C code I've posted in previous msg.)

Then dentry_cache grows really fast.

But if within 'C' code if I write while(1), then everything is okay.
(No substantial growth ).

Thanks
-Kunal

On 12/6/06, Kunal Trivedi <[EMAIL PROTECTED]> wrote:

Hi,
I am running 2.6.18 kernel on 64 bit AMD machine. (procinfo attached
at the end of the mail). One of the things I have noticed that
dentry_cache grows really fast under certain code path.
So, far I have not seen any problems, but I would like to get some
more input on this subject. Is it okay for dentry_cache to grow that
much ?

I've run following program for an ~1.00 hour. And my slabinfo shows following.

# cat /proc/slabinfo | grep dentry
dentry_cache  5228333 5228333224   171 : tunables  120
608 : slabdata 307549 307549  0


# cat /proc/meminfo
MemTotal:  8173080 kB
MemFree:   6787852 kB
Buffers: 42048 kB
Cached:  72616 kB
SwapCached:  0 kB
Active:  88608 kB
Inactive:29796 kB
HighTotal:   0 kB
HighFree:0 kB
LowTotal:  8173080 kB
LowFree:   6787852 kB
SwapTotal: 2096472 kB
SwapFree:  2096472 kB
Dirty:  48 kB
Writeback:   0 kB
AnonPages:3716 kB
Mapped:   3336 kB
Slab:  1251292 kB
PageTables:192 kB
NFS_Unstable:0 kB
Bounce:  0 kB
CommitLimit:  10269552 kB
Committed_AS:11500 kB
VmallocTotal: 34359738367 kB
VmallocUsed:  1272 kB
VmallocChunk: 34359737015 kB
HugePages_Total: 0
HugePages_Free:  0
HugePages_Rsvd:  0
Hugepagesize: 2048 kB


int
main()
{
int fd;
char fname[] = "/tmp/proc-output-XX";

fd = mkstemp(fname);
close(fd);
unlink(fname);
return 0;
}

Please advice,

Thanks
-Kunal


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

dentry cache grows to really large number on 64bit machine

2006-12-06 Thread Kunal Trivedi 

Hi,
I am running 2.6.18 kernel on 64 bit AMD machine. (procinfo attached
at the end of the mail). One of the things I have noticed that
dentry_cache grows really fast under certain code path.
So, far I have not seen any problems, but I would like to get some
more input on this subject. Is it okay for dentry_cache to grow that
much ?

I've run following program for an ~1.00 hour. And my slabinfo shows following.

# cat /proc/slabinfo | grep dentry
dentry_cache  5228333 5228333224   171 : tunables  120
608 : slabdata 307549 307549  0


# cat /proc/meminfo
MemTotal:  8173080 kB
MemFree:   6787852 kB
Buffers: 42048 kB
Cached:  72616 kB
SwapCached:  0 kB
Active:  88608 kB
Inactive:29796 kB
HighTotal:   0 kB
HighFree:0 kB
LowTotal:  8173080 kB
LowFree:   6787852 kB
SwapTotal: 2096472 kB
SwapFree:  2096472 kB
Dirty:  48 kB
Writeback:   0 kB
AnonPages:3716 kB
Mapped:   3336 kB
Slab:  1251292 kB
PageTables:192 kB
NFS_Unstable:0 kB
Bounce:  0 kB
CommitLimit:  10269552 kB
Committed_AS:11500 kB
VmallocTotal: 34359738367 kB
VmallocUsed:  1272 kB
VmallocChunk: 34359737015 kB
HugePages_Total: 0
HugePages_Free:  0
HugePages_Rsvd:  0
Hugepagesize: 2048 kB


int
main()
{
   int fd;
   char fname[] = "/tmp/proc-output-XX";

   fd = mkstemp(fname);
   close(fd);
   unlink(fname);
   return 0;
}

Please advice,

Thanks
-Kunal
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: Possible Bug in VM accounting (Committed_AS) on x86_64 architecture ?

2006-11-29 Thread Kunal Trivedi 

Hi,
I am running into a problem and due to limited understanding unable to
solve it...
Problem:
-
On 64 bit machines (running linux 2.6.xx), Committed_AS grows over the
period of time. Within 3-4 weeks system reaches a stage where further
malloc() returns -ENOMEM. Test shows that just running simple program
(which malloc, touch, free memory) cause this increase in Committed_AS
number.  (I am using vm-overcommit... Below detailed information)

System/Kernel Spec:
--
-
CentOS kernel, 2.6.9-34.EL-x86_64_SMP
Physical: 8G
Swap: 2G
Machine type: AMD
Nothing un-usual in .config. Pretty much using standard options. (If
needed I will send .config)
Here is /proc/meminfo numbers:

MemTotal:  8169736 kB
MemFree:   1256676 kB
Buffers:123496 kB
Cached:5009620 kB
SwapCached:  0 kB
Active:5297288 kB
Inactive:  1003628 kB
HighTotal:   0 kB
HighFree:0 kB
LowTotal:  8169736 kB
LowFree:   1256676 kB
SwapTotal: 2096472 kB
SwapFree:  2096472 kB
Dirty:   10036 kB
Writeback:   0 kB
Mapped:5814464 kB
Slab:   571352 kB
Committed_AS:  7125024 kB
PageTables:  12916 kB
VmallocTotal: 536870911 kB
VmallocUsed:268300 kB
VmallocChunk: 536601679 kB
HugePages_Total: 0
HugePages_Free:  0
Hugepagesize: 2048 kB

OverCommit Options:
-
vm.overcommit_ratio = 100
vm.overcommit_memory = 2

Hence my virtual limit is 10G (8*100/100 + 2)

Detailed Description:
---
I have noticed that 64 bit machine with overcommit policy (as above)
starts giving problem within 3-4 weeks. To prove that I've written
small program.
It allocates memory of different sizes (not that it matters much due
to caching of diffeent malloc. I am using standard ptmalloc). Sizes
are 16B, 32B, 64B, 256B, 1024B, 57B, 127B and so on... . Then it
touches that memory (memset) and then free it. These operations are
being performed in while(1) loop.

Observations:
--
Committed_AS: number gorws 5M per hour. I made sure that nothing else
is running on the system during that time...

Is there any obvious problem reported for vm accounting on 64 bit
architecture ? Or this is expected ? Or vm-overcommit is only meant
for 32 bit architecture ?

Please advice..

Thanks in advance.
--

-Kunal
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/