Re: 2.6.20-rc6-mm1

2007-01-29 Thread Ingo Molnar

* Herbert Xu [EMAIL PROTECTED] wrote:

 Actually, maybe I was confusing this with the fixes Ingo had for 
 local_bh_disable vs. preemption in the -rt tree.  Ingo, do you have 
 preemptible RCU support in your -rt tree and if so did you have to fix 
 the networking stack to behave correctly with it?

yeah, -rt is the main prototyping tree for PREEMPT_RCU, and it has been 
included in -rt for 1.5 years or so. There were only some small things 
here and there, but with Paul's latest design i dont remember anything 
but the occasional place that needs to be marked raw_smp_processor_id(). 
CONFIG_DEBUG_PREEMPT ought to catch those. I dont remember any big 
breakage.

i've just reviewed all changes to net/* in -rt, and the changes below 
are the ones that seem to be upstream-relevant.

Ingo

Index: linux/net/ipv4/multipath_wrandom.c
===
--- linux.orig/net/ipv4/multipath_wrandom.c
+++ linux/net/ipv4/multipath_wrandom.c
@@ -301,6 +301,7 @@ static void wrandom_flush(void)
for (i = 0; i  MULTIPATH_STATE_SIZE; ++i) {
struct multipath_route *r;
 
+   rcu_read_lock();
spin_lock_bh(state[i].lock);
list_for_each_entry_rcu(r, state[i].head, list) {
struct multipath_dest *d;
@@ -315,6 +316,7 @@ static void wrandom_flush(void)
}
 
spin_unlock_bh(state[i].lock);
+   rcu_read_unlock();
}
 }
 
Index: linux/net/ipv4/netfilter/arp_tables.c
===
--- linux.orig/net/ipv4/netfilter/arp_tables.c
+++ linux/net/ipv4/netfilter/arp_tables.c
@@ -245,7 +245,7 @@ unsigned int arpt_do_table(struct sk_buf
 
read_lock_bh(table-lock);
private = table-private;
-   table_base = (void *)private-entries[smp_processor_id()];
+   table_base = (void *)private-entries[raw_smp_processor_id()];
e = get_entry(table_base, private-hook_entry[hook]);
back = get_entry(table_base, private-underflow[hook]);
 
@@ -955,7 +955,7 @@ static int do_add_counters(void __user *
 
i = 0;
/* Choose the copy that is on our node */
-   loc_cpu_entry = private-entries[smp_processor_id()];
+   loc_cpu_entry = private-entries[raw_smp_processor_id()];
ARPT_ENTRY_ITERATE(loc_cpu_entry,
   private-size,
   add_counter_to_entry,
Index: linux/net/ipv4/netfilter/ip_tables.c
===
--- linux.orig/net/ipv4/netfilter/ip_tables.c
+++ linux/net/ipv4/netfilter/ip_tables.c
@@ -246,7 +246,7 @@ ipt_do_table(struct sk_buff **pskb,
read_lock_bh(table-lock);
IP_NF_ASSERT(table-valid_hooks  (1  hook));
private = table-private;
-   table_base = (void *)private-entries[smp_processor_id()];
+   table_base = (void *)private-entries[raw_smp_processor_id()];
e = get_entry(table_base, private-hook_entry[hook]);
 
/* For return from builtin chain */
Index: linux/net/ipv4/tcp.c
===
--- linux.orig/net/ipv4/tcp.c
+++ linux/net/ipv4/tcp.c
@@ -1138,10 +1138,10 @@ int tcp_recvmsg(struct kiocb *iocb, stru
preempt_disable();
if ((len  sysctl_tcp_dma_copybreak)  !(flags  MSG_PEEK) 
!sysctl_tcp_low_latency  __get_cpu_var(softnet_data).net_dma) {
-   preempt_enable_no_resched();
+   preempt_enable();
tp-ucopy.pinned_list = dma_pin_iovec_pages(msg-msg_iov, len);
} else
-   preempt_enable_no_resched();
+   preempt_enable();
 #endif
 
do {
Index: linux/net/ipv6/netfilter/ip6_tables.c
===
--- linux.orig/net/ipv6/netfilter/ip6_tables.c
+++ linux/net/ipv6/netfilter/ip6_tables.c
@@ -282,7 +282,7 @@ ip6t_do_table(struct sk_buff **pskb,
read_lock_bh(table-lock);
private = table-private;
IP_NF_ASSERT(table-valid_hooks  (1  hook));
-   table_base = (void *)private-entries[smp_processor_id()];
+   table_base = (void *)private-entries[raw_smp_processor_id()];
e = get_entry(table_base, private-hook_entry[hook]);
 
/* For return from builtin chain */
@@ -1097,7 +1097,7 @@ do_add_counters(void __user *user, unsig
 
i = 0;
/* Choose the copy that is on our node */
-   loc_cpu_entry = private-entries[smp_processor_id()];
+   loc_cpu_entry = private-entries[raw_smp_processor_id()];
IP6T_ENTRY_ITERATE(loc_cpu_entry,
  private-size,
  add_counter_to_entry,
Index: linux/net/xfrm/xfrm_user.c
===
--- linux.orig/net/xfrm/xfrm_user.c
+++ linux/net/xfrm/xfrm_user.c
@@ -1273,13 +1273,12 @@ static int 

Re: 2.6.20-rc6-mm1

2007-01-28 Thread Michal Piotrowski
Andrew Morton napisaƂ(a):
 Temporarily at
 
   http://userweb.kernel.org/~akpm/2.6.20-rc6-mm1/
 
 will appear one day at
 
   
 ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.20-rc6/2.6.20-rc6-mm1/
 
 

Jan 28 22:58:29 euridica kernel: BUG: using smp_processor_id() in preemptible 
[0001] code: yum-updatesd/2846
Jan 28 22:58:29 euridica kernel: caller is nf_conntrack_in+0x363/0x47f 
[nf_conntrack]
Jan 28 22:58:29 euridica kernel:  [c01053c6] show_trace_log_lvl+0x1a/0x2f
Jan 28 22:58:29 euridica kernel:  [c0105ad6] show_trace+0x12/0x14
Jan 28 22:58:29 euridica kernel:  [c0105b98] dump_stack+0x16/0x18
Jan 28 22:58:29 euridica kernel:  [c0207803] debug_smp_processor_id+0xb3/0xc8
Jan 28 22:58:29 euridica kernel:  [fdbf8ad0] nf_conntrack_in+0x363/0x47f 
[nf_conntrack]
Jan 28 22:58:29 euridica kernel:  [fd9c32c4] ipv4_conntrack_local+0x53/0x5b 
[nf_conntrack_ipv4]
Jan 28 22:58:29 euridica kernel:  [c02f2286] nf_iterate+0x36/0x67
Jan 28 22:58:29 euridica kernel:  [c02f241b] nf_hook_slow+0x52/0xbe
Jan 28 22:58:29 euridica kernel:  [c02fd280] ip_queue_xmit+0x3d1/0x420
Jan 28 22:58:29 euridica kernel:  [c030b9b9] tcp_transmit_skb+0x78b/0x7ca
Jan 28 22:58:29 euridica kernel:  [c030d452] 
__tcp_push_pending_frames+0x732/0x811
Jan 28 22:58:29 euridica kernel:  [c030e3de] tcp_send_fin+0x146/0x150
Jan 28 22:58:29 euridica kernel:  [c03042c9] tcp_close+0x22b/0x556
Jan 28 22:58:29 euridica kernel:  [c031d595] inet_release+0x43/0x49
Jan 28 22:58:29 euridica kernel:  [c02d74c8] sock_release+0x17/0x9d
Jan 28 22:58:29 euridica kernel:  [c02d7843] sock_close+0x2d/0x33
Jan 28 22:58:29 euridica kernel:  [c0178a0a] __fput+0xca/0x198
Jan 28 22:58:29 euridica kernel:  [c0178c44] fput+0x32/0x36
Jan 28 22:58:29 euridica kernel:  [c017606a] filp_close+0x54/0x5c
Jan 28 22:58:29 euridica kernel:  [c0124e72] put_files_struct+0x7d/0xba
Jan 28 22:58:29 euridica kernel:  [c012625e] do_exit+0x30d/0x8e6
Jan 28 22:58:29 euridica kernel:  [c01268b9] sys_exit_group+0x0/0x11
Jan 28 22:58:29 euridica kernel:  [c012f177] get_signal_to_deliver+0x734/0x760
Jan 28 22:58:29 euridica kernel:  [c010391e] do_notify_resume+0x94/0x749
Jan 28 22:58:29 euridica kernel:  [c01044a1] work_notifysig+0x13/0x1a
Jan 28 22:58:29 euridica kernel:  ===

http://www.stardust.webpages.pl/files/tbf/euridica/2.6.20-rc6-mm1/mm-config

Regards,
Michal

-- 
Michal K. K. Piotrowski
LTG - Linux Testers Group
(http://www.stardust.webpages.pl/ltg/)
-
To unsubscribe from this list: send the line unsubscribe netdev in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: 2.6.20-rc6-mm1

2007-01-28 Thread Andrew Morton
On Sun, 28 Jan 2007 23:31:00 +0100
Michal Piotrowski [EMAIL PROTECTED] wrote:

 Jan 28 22:58:29 euridica kernel: BUG: using smp_processor_id() in preemptible 
 [0001] code: yum-updatesd/2846
 Jan 28 22:58:29 euridica kernel: caller is nf_conntrack_in+0x363/0x47f 
 [nf_conntrack]

I'll plug that with this:


diff -puN include/net/netfilter/nf_conntrack.h~netfilter-warning-fix 
include/net/netfilter/nf_conntrack.h
--- a/include/net/netfilter/nf_conntrack.h~netfilter-warning-fix
+++ a/include/net/netfilter/nf_conntrack.h
@@ -254,7 +254,12 @@ extern atomic_t nf_conntrack_count;
 extern int nf_conntrack_max;
 
 DECLARE_PER_CPU(struct ip_conntrack_stat, nf_conntrack_stat);
-#define NF_CT_STAT_INC(count) (__get_cpu_var(nf_conntrack_stat).count++)
+#define NF_CT_STAT_INC(count)  \
+   do {\
+   preempt_disable();  \
+   __get_cpu_var(nf_conntrack_stat).count++;   \
+   preempt_enable();   \
+   } while (0);
 
 /* no helper, no nat */
 #defineNF_CT_F_BASIC   0
_

and will let others worry about what the real fix (for 2.6.20, please) is.
-
To unsubscribe from this list: send the line unsubscribe netdev in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: 2.6.20-rc6-mm1

2007-01-28 Thread Herbert Xu
Michal Piotrowski [EMAIL PROTECTED] wrote:

 Jan 28 22:58:29 euridica kernel: BUG: using smp_processor_id() in preemptible 
 [0001] code: yum-updatesd/2846
 Jan 28 22:58:29 euridica kernel: caller is nf_conntrack_in+0x363/0x47f 
 [nf_conntrack]
 Jan 28 22:58:29 euridica kernel:  [c01053c6] show_trace_log_lvl+0x1a/0x2f
 Jan 28 22:58:29 euridica kernel:  [c0105ad6] show_trace+0x12/0x14
 Jan 28 22:58:29 euridica kernel:  [c0105b98] dump_stack+0x16/0x18
 Jan 28 22:58:29 euridica kernel:  [c0207803] 
 debug_smp_processor_id+0xb3/0xc8
 Jan 28 22:58:29 euridica kernel:  [fdbf8ad0] nf_conntrack_in+0x363/0x47f 
 [nf_conntrack]
 Jan 28 22:58:29 euridica kernel:  [fd9c32c4] ipv4_conntrack_local+0x53/0x5b 
 [nf_conntrack_ipv4]
 Jan 28 22:58:29 euridica kernel:  [c02f2286] nf_iterate+0x36/0x67
 Jan 28 22:58:29 euridica kernel:  [c02f241b] nf_hook_slow+0x52/0xbe

This shouldn't have happened.  nf_hook_slow calls nf_iterate and
therefore everything under it with preemption disabled.  So something
must've reenabled it before hitting nf_conntrack_in.

Cheers,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmVHI~} [EMAIL PROTECTED]
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
-
To unsubscribe from this list: send the line unsubscribe netdev in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: 2.6.20-rc6-mm1

2007-01-28 Thread Herbert Xu
On Mon, Jan 29, 2007 at 04:17:44PM +1100, Herbert Xu wrote:
 Michal Piotrowski [EMAIL PROTECTED] wrote:
 
  Jan 28 22:58:29 euridica kernel: BUG: using smp_processor_id() in 
  preemptible [0001] code: yum-updatesd/2846
  Jan 28 22:58:29 euridica kernel: caller is nf_conntrack_in+0x363/0x47f 
  [nf_conntrack]
  Jan 28 22:58:29 euridica kernel:  [c01053c6] show_trace_log_lvl+0x1a/0x2f
  Jan 28 22:58:29 euridica kernel:  [c0105ad6] show_trace+0x12/0x14
  Jan 28 22:58:29 euridica kernel:  [c0105b98] dump_stack+0x16/0x18
  Jan 28 22:58:29 euridica kernel:  [c0207803] 
  debug_smp_processor_id+0xb3/0xc8
  Jan 28 22:58:29 euridica kernel:  [fdbf8ad0] nf_conntrack_in+0x363/0x47f 
  [nf_conntrack]
  Jan 28 22:58:29 euridica kernel:  [fd9c32c4] 
  ipv4_conntrack_local+0x53/0x5b [nf_conntrack_ipv4]
  Jan 28 22:58:29 euridica kernel:  [c02f2286] nf_iterate+0x36/0x67
  Jan 28 22:58:29 euridica kernel:  [c02f241b] nf_hook_slow+0x52/0xbe
 
 This shouldn't have happened.  nf_hook_slow calls nf_iterate and
 therefore everything under it with preemption disabled.  So something
 must've reenabled it before hitting nf_conntrack_in.

Does mm now have the preemptible RCU stuff? If so that would certainly
explain this.

IIRC Ingo had made fixes for the networking stack in his rt tree since
the networking code assumes in lots of places that rcu_read_lock
disables preemption.

Cheers,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmVHI~} [EMAIL PROTECTED]
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
-
To unsubscribe from this list: send the line unsubscribe netdev in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: 2.6.20-rc6-mm1

2007-01-28 Thread Andrew Morton
On Mon, 29 Jan 2007 16:29:29 +1100
Herbert Xu [EMAIL PROTECTED] wrote:

 On Mon, Jan 29, 2007 at 04:17:44PM +1100, Herbert Xu wrote:
  Michal Piotrowski [EMAIL PROTECTED] wrote:
  
   Jan 28 22:58:29 euridica kernel: BUG: using smp_processor_id() in 
   preemptible [0001] code: yum-updatesd/2846
   Jan 28 22:58:29 euridica kernel: caller is nf_conntrack_in+0x363/0x47f 
   [nf_conntrack]
   Jan 28 22:58:29 euridica kernel:  [c01053c6] 
   show_trace_log_lvl+0x1a/0x2f
   Jan 28 22:58:29 euridica kernel:  [c0105ad6] show_trace+0x12/0x14
   Jan 28 22:58:29 euridica kernel:  [c0105b98] dump_stack+0x16/0x18
   Jan 28 22:58:29 euridica kernel:  [c0207803] 
   debug_smp_processor_id+0xb3/0xc8
   Jan 28 22:58:29 euridica kernel:  [fdbf8ad0] 
   nf_conntrack_in+0x363/0x47f [nf_conntrack]
   Jan 28 22:58:29 euridica kernel:  [fd9c32c4] 
   ipv4_conntrack_local+0x53/0x5b [nf_conntrack_ipv4]
   Jan 28 22:58:29 euridica kernel:  [c02f2286] nf_iterate+0x36/0x67
   Jan 28 22:58:29 euridica kernel:  [c02f241b] nf_hook_slow+0x52/0xbe
  
  This shouldn't have happened.  nf_hook_slow calls nf_iterate and
  therefore everything under it with preemption disabled.  So something
  must've reenabled it before hitting nf_conntrack_in.
 
 Does mm now have the preemptible RCU stuff? If so that would certainly
 explain this.

It does,

 IIRC Ingo had made fixes for the networking stack in his rt tree since
 the networking code assumes in lots of places that rcu_read_lock
 disables preemption.

oh.  We'd better find those fixes then.  I wonder what other code made that
(rather hacky) assumption?  I guess we have enough debug stuff in there to
find out..
-
To unsubscribe from this list: send the line unsubscribe netdev in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: 2.6.20-rc6-mm1

2007-01-28 Thread Herbert Xu
On Sun, Jan 28, 2007 at 10:43:12PM -0800, Andrew Morton wrote:

  IIRC Ingo had made fixes for the networking stack in his rt tree since
  the networking code assumes in lots of places that rcu_read_lock
  disables preemption.
 
 oh.  We'd better find those fixes then.  I wonder what other code made that
 (rather hacky) assumption?  I guess we have enough debug stuff in there to
 find out..

Actually, maybe I was confusing this with the fixes Ingo had for
local_bh_disable vs. preemption in the -rt tree.  Ingo, do you
have preemptible RCU support in your -rt tree and if so did you
have to fix the networking stack to behave correctly with it?

It could also be that the fixes for local_bh_disable also masked
any problems that would trigger under preemptible RCU.

Cheers,
-- 
Visit Openswan at http://www.openswan.org/
Email: Herbert Xu ~{PmVHI~} [EMAIL PROTECTED]
Home Page: http://gondor.apana.org.au/~herbert/
PGP Key: http://gondor.apana.org.au/~herbert/pubkey.txt
-
To unsubscribe from this list: send the line unsubscribe netdev in
the body of a message to [EMAIL PROTECTED]
More majordomo info at  http://vger.kernel.org/majordomo-info.html