Module Name: src
Committed By: ozaki-r
Date: Mon Dec 12 03:55:58 UTC 2016
Modified Files:
src/sys/net: if.c if.h if_faith.c if_mpls.c if_stf.c radix.c route.c
route.h rtbl.c rtsock.c
src/sys/netinet: if_arp.c if_atm.c in.c in_gif.c in_pcb.c ip_carp.c
ip_icmp.c ip_input.c ip_output.c
src/sys/netinet6: icmp6.c in6.c in6_gif.c in6_pcb.c nd6.c nd6_nbr.c
Log Message:
Make the routing table and rtcaches MP-safe
See the following descriptions for details.
Proposed on tech-kern and tech-net
Overview
--------
We protect the routing table with a rwock and protect
rtcaches with another rwlock. Each rtentry is protected
from being freed or updated via reference counting and psref.
Global rwlocks
--------------
There are two rwlocks; one for the routing table (rt_lock) and
the other for rtcaches (rtcache_lock). rtcache_lock covers
all existing rtcaches; there may have room for optimizations
(future work).
The locking order is rtcache_lock first and rt_lock is next.
rtentry references
------------------
References to an rtentry is managed with reference counting
and psref. Either of the two mechanisms is used depending on
where a rtentry is obtained. Reference counting is used when
we obtain a rtentry from the routing table directly via
rtalloc1 and rtrequest{,1} while psref is used when we obtain
a rtentry from a rtcache via rtcache_* APIs. In both cases,
a caller can sleep/block with holding an obtained rtentry.
The reasons why we use two different mechanisms are (i) only
using reference counting hurts the performance due to atomic
instructions (rtcache case) (ii) ease of implementation;
applying psref to APIs such rtaloc1 and rtrequest{,1} requires
additional works (adding a local variable and an argument).
We will finally migrate to use only psref but we can do it
when we have a lockless routing table alternative.
Reference counting for rtentry
------------------------------
rt_refcnt now doesn't count permanent references such as for
rt_timers and rtcaches, instead it is used only for temporal
references when obtaining a rtentry via rtalloc1 and rtrequest{,1}.
We can do so because destroying a rtentry always involves
removing references of rt_timers and rtcaches to the rtentry
and we don't need to track such references. This also makes
it easy to wait for readers to release references on deleting
or updating a rtentry, i.e., we can simply wait until the
reference counter is 0 or 1. (If there are permanent references
the counter can be arbitrary.)
rt_ref increments a reference counter of a rtentry and rt_unref
decrements it. rt_ref is called inside APIs (rtalloc1 and
rtrequest{,1} so users don't need to care about it while
users must call rt_unref to an obtained rtentry after using it.
rtfree is removed and we use rt_unref and rt_free instead.
rt_unref now just decrements the counter of a given rtentry
and rt_free just tries to destroy a given rtentry.
See the next section for destructions of rtentries by rt_free.
Destructions of rtentries
-------------------------
We destroy a rtentry only when we call rtrequst{,1}(RTM_DELETE);
the original implementation can destroy in any rtfree where it's
the last reference. If we use reference counting or psref, it's
easy to understand if the place that a rtentry is destroyed is
fixed.
rt_free waits for references to a given rtentry to be released
before actually destroying the rtentry. rt_free uses a condition
variable (cv_wait) (and psref_target_destroy for psref) to wait.
Unfortunately rtrequst{,1}(RTM_DELETE) can be called in softint
that we cannot use cv_wait. In that case, we have to defer the
destruction to a workqueue.
rtentry#rt_cv, rtentry#rt_psref and global variables
(see rt_free_global) are added to conduct the procedure.
Updates of rtentries
--------------------
One difficulty to use refcnt/psref instead of rwlock for rtentry
is updates of rtentries. We need an additional mechanism to
prevent readers from seeing inconsistency of a rtentry being
updated.
We introduce RTF_UPDATING flag to rtentries that are updating.
While the flag is set to a rtentry, users cannot acquire the
rtentry. By doing so, we avoid users to see inconsistent
rtentries.
There are two options when a user tries to acquire a rtentry
with the RTF_UPDATING flag; if a user runs in softint context
the user fails to acquire a rtentry (NULL is returned).
Otherwise a user waits until the update completes by waiting
on cv.
The procedure of a updater is simpler to destruction of
a rtentry. Wait on cv (and psref) and after all readers left,
proceed with the update.
Global variables (see rt_update_global) are added to conduct
the procedure.
Currently we apply the mechanism to only RTM_CHANGE in
rtsock.c. We would have to apply other codes. See
"Known issues" section.
psref for rtentry
-----------------
When we obtain a rtentry from a rtcache via rtcache_* APIs,
psref is used to reference to the rtentry.
rtcache_ref acquires a reference to a rtentry with psref
and rtcache_unref releases the reference after using it.
rtcache_ref is called inside rtcache_* APIs and users don't
need to take care of it while users must call rtcache_unref
to release the reference.
struct psref and int bound that is needed for psref is
embedded into struct route. By doing so we don't need to
add local variables and additional argument to APIs.
However this adds another constraint to psref other than
reference counting one's; holding a reference of an rtentry
via a rtcache is allowed by just one caller at the same time.
So we must not acquire a rtentry via a rtcache twice and
avoid a recursive use of a rtcache. And also a rtcache must
be arranged to be used by a LWP/softint at the same time
somehow. For IP forwarding case, we have per-CPU rtcaches
used in softint so the constraint is guaranteed. For a h
rtcache of a PCB case, the constraint is guaranteed by the
solock of each PCB. Any other cases (pf, ipf, stf and ipsec)
are currently guaranteed by only the existence of the global
locks (softnet_lock and/or KERNEL_LOCK). If we've found the
cases that we cannot guarantee the constraint, we would need
to introduce other rtcache APIs that use simple reference
counting.
psref of rtcache is created with IPL_SOFTNET and so rtcache
shouldn't used at an IPL higher than IPL_SOFTNET.
Note that rtcache_free is used to invalidate a given rtcache.
We don't need another care by my change; just keep them as
they are.
Performance impact
------------------
When NET_MPSAFE is disabled the performance drop is 3% while
when it's enabled the drop is increased to 11%. The difference
comes from that currently we don't take any global locks and
don't use psref if NET_MPSAFE is disabled.
We can optimize the performance of the case of NET_MPSAFE
on by reducing lookups of rtcache that uses psref;
currently we do two lookups but we should be able to trim
one of two. This is a future work.
Known issues
------------
There are two known issues to be solved; one is that
a caller of rtrequest(RTM_ADD) may change rtentry (see rtinit).
We need to prevent new references during the update. Or
we may be able to remove the code (perhaps, need more
investigations).
The other is rtredirect that updates a rtentry. We need
to apply our update mechanism, however it's not easy because
rtredirect is called in softint and we cannot apply our
mechanism simply. One solution is to defer rtredirect to
a workqueue but it requires some code restructuring.
To generate a diff of this commit:
cvs rdiff -u -r1.365 -r1.366 src/sys/net/if.c
cvs rdiff -u -r1.230 -r1.231 src/sys/net/if.h
cvs rdiff -u -r1.54 -r1.55 src/sys/net/if_faith.c
cvs rdiff -u -r1.28 -r1.29 src/sys/net/if_mpls.c
cvs rdiff -u -r1.100 -r1.101 src/sys/net/if_stf.c
cvs rdiff -u -r1.46 -r1.47 src/sys/net/radix.c
cvs rdiff -u -r1.182 -r1.183 src/sys/net/route.c
cvs rdiff -u -r1.108 -r1.109 src/sys/net/route.h
cvs rdiff -u -r1.4 -r1.5 src/sys/net/rtbl.c
cvs rdiff -u -r1.198 -r1.199 src/sys/net/rtsock.c
cvs rdiff -u -r1.232 -r1.233 src/sys/netinet/if_arp.c
cvs rdiff -u -r1.38 -r1.39 src/sys/netinet/if_atm.c
cvs rdiff -u -r1.190 -r1.191 src/sys/netinet/in.c
cvs rdiff -u -r1.84 -r1.85 src/sys/netinet/in_gif.c
cvs rdiff -u -r1.171 -r1.172 src/sys/netinet/in_pcb.c
cvs rdiff -u -r1.79 -r1.80 src/sys/netinet/ip_carp.c
cvs rdiff -u -r1.153 -r1.154 src/sys/netinet/ip_icmp.c
cvs rdiff -u -r1.346 -r1.347 src/sys/netinet/ip_input.c
cvs rdiff -u -r1.264 -r1.265 src/sys/netinet/ip_output.c
cvs rdiff -u -r1.202 -r1.203 src/sys/netinet6/icmp6.c
cvs rdiff -u -r1.223 -r1.224 src/sys/netinet6/in6.c
cvs rdiff -u -r1.80 -r1.81 src/sys/netinet6/in6_gif.c
cvs rdiff -u -r1.153 -r1.154 src/sys/netinet6/in6_pcb.c
cvs rdiff -u -r1.215 -r1.216 src/sys/netinet6/nd6.c
cvs rdiff -u -r1.131 -r1.132 src/sys/netinet6/nd6_nbr.c
Please note that diffs are not public domain; they are subject to the
copyright notices on the relevant files.
Modified files:
Index: src/sys/net/if.c
diff -u src/sys/net/if.c:1.365 src/sys/net/if.c:1.366
--- src/sys/net/if.c:1.365 Fri Dec 9 02:38:14 2016
+++ src/sys/net/if.c Mon Dec 12 03:55:57 2016
@@ -1,4 +1,4 @@
-/* $NetBSD: if.c,v 1.365 2016/12/09 02:38:14 christos Exp $ */
+/* $NetBSD: if.c,v 1.366 2016/12/12 03:55:57 ozaki-r Exp $ */
/*-
* Copyright (c) 1999, 2000, 2001, 2008 The NetBSD Foundation, Inc.
@@ -90,7 +90,7 @@
*/
#include <sys/cdefs.h>
-__KERNEL_RCSID(0, "$NetBSD: if.c,v 1.365 2016/12/09 02:38:14 christos Exp $");
+__KERNEL_RCSID(0, "$NetBSD: if.c,v 1.366 2016/12/12 03:55:57 ozaki-r Exp $");
#if defined(_KERNEL_OPT)
#include "opt_inet.h"
@@ -1686,6 +1686,7 @@ ifa_psref_init(struct ifaddr *ifa)
void
ifaref(struct ifaddr *ifa)
{
+ KASSERT(!ISSET(ifa->ifa_flags, IFA_DESTROYING));
ifa->ifa_refcnt++;
}
@@ -1700,6 +1701,13 @@ ifafree(struct ifaddr *ifa)
}
}
+bool
+ifa_is_destroying(struct ifaddr *ifa)
+{
+
+ return ISSET(ifa->ifa_flags, IFA_DESTROYING);
+}
+
void
ifa_insert(struct ifnet *ifp, struct ifaddr *ifa)
{
Index: src/sys/net/if.h
diff -u src/sys/net/if.h:1.230 src/sys/net/if.h:1.231
--- src/sys/net/if.h:1.230 Thu Dec 8 01:06:35 2016
+++ src/sys/net/if.h Mon Dec 12 03:55:57 2016
@@ -1,4 +1,4 @@
-/* $NetBSD: if.h,v 1.230 2016/12/08 01:06:35 ozaki-r Exp $ */
+/* $NetBSD: if.h,v 1.231 2016/12/12 03:55:57 ozaki-r Exp $ */
/*-
* Copyright (c) 1999, 2000, 2001 The NetBSD Foundation, Inc.
@@ -604,6 +604,7 @@ struct ifaddr {
#endif
};
#define IFA_ROUTE RTF_UP /* (0x01) route installed */
+#define IFA_DESTROYING 0x2
/*
* Message format for use in obtaining information about interfaces from
@@ -1001,6 +1002,7 @@ void ifa_psref_init(struct ifaddr *);
void ifa_acquire(struct ifaddr *, struct psref *);
void ifa_release(struct ifaddr *, struct psref *);
bool ifa_held(struct ifaddr *);
+bool ifa_is_destroying(struct ifaddr *);
void ifaref(struct ifaddr *);
void ifafree(struct ifaddr *);
Index: src/sys/net/if_faith.c
diff -u src/sys/net/if_faith.c:1.54 src/sys/net/if_faith.c:1.55
--- src/sys/net/if_faith.c:1.54 Sun Aug 7 17:38:33 2016
+++ src/sys/net/if_faith.c Mon Dec 12 03:55:57 2016
@@ -1,4 +1,4 @@
-/* $NetBSD: if_faith.c,v 1.54 2016/08/07 17:38:33 christos Exp $ */
+/* $NetBSD: if_faith.c,v 1.55 2016/12/12 03:55:57 ozaki-r Exp $ */
/* $KAME: if_faith.c,v 1.21 2001/02/20 07:59:26 itojun Exp $ */
/*
@@ -40,7 +40,7 @@
*/
#include <sys/cdefs.h>
-__KERNEL_RCSID(0, "$NetBSD: if_faith.c,v 1.54 2016/08/07 17:38:33 christos Exp $");
+__KERNEL_RCSID(0, "$NetBSD: if_faith.c,v 1.55 2016/12/12 03:55:57 ozaki-r Exp $");
#ifdef _KERNEL_OPT
#include "opt_inet.h"
@@ -322,7 +322,7 @@ faithprefix(struct in6_addr *in6)
else
ret = 0;
if (rt)
- rtfree(rt);
+ rt_unref(rt);
return ret;
}
#endif
Index: src/sys/net/if_mpls.c
diff -u src/sys/net/if_mpls.c:1.28 src/sys/net/if_mpls.c:1.29
--- src/sys/net/if_mpls.c:1.28 Mon Oct 3 11:06:06 2016
+++ src/sys/net/if_mpls.c Mon Dec 12 03:55:57 2016
@@ -1,4 +1,4 @@
-/* $NetBSD: if_mpls.c,v 1.28 2016/10/03 11:06:06 ozaki-r Exp $ */
+/* $NetBSD: if_mpls.c,v 1.29 2016/12/12 03:55:57 ozaki-r Exp $ */
/*
* Copyright (c) 2010 The NetBSD Foundation, Inc.
@@ -30,7 +30,7 @@
*/
#include <sys/cdefs.h>
-__KERNEL_RCSID(0, "$NetBSD: if_mpls.c,v 1.28 2016/10/03 11:06:06 ozaki-r Exp $");
+__KERNEL_RCSID(0, "$NetBSD: if_mpls.c,v 1.29 2016/12/12 03:55:57 ozaki-r Exp $");
#ifdef _KERNEL_OPT
#include "opt_inet.h"
@@ -304,7 +304,7 @@ mpls_output(struct ifnet *ifp, struct mb
}
err = mpls_send_frame(m, rt1->rt_ifp, rt);
- rtfree(rt1);
+ rt_unref(rt1);
return err;
}
@@ -481,7 +481,7 @@ done:
if (error != 0 && m != NULL)
m_freem(m);
if (rt != NULL)
- rtfree(rt);
+ rt_unref(rt);
return error;
}
Index: src/sys/net/if_stf.c
diff -u src/sys/net/if_stf.c:1.100 src/sys/net/if_stf.c:1.101
--- src/sys/net/if_stf.c:1.100 Thu Dec 8 05:16:33 2016
+++ src/sys/net/if_stf.c Mon Dec 12 03:55:57 2016
@@ -1,4 +1,4 @@
-/* $NetBSD: if_stf.c,v 1.100 2016/12/08 05:16:33 ozaki-r Exp $ */
+/* $NetBSD: if_stf.c,v 1.101 2016/12/12 03:55:57 ozaki-r Exp $ */
/* $KAME: if_stf.c,v 1.62 2001/06/07 22:32:16 itojun Exp $ */
/*
@@ -75,7 +75,7 @@
*/
#include <sys/cdefs.h>
-__KERNEL_RCSID(0, "$NetBSD: if_stf.c,v 1.100 2016/12/08 05:16:33 ozaki-r Exp $");
+__KERNEL_RCSID(0, "$NetBSD: if_stf.c,v 1.101 2016/12/12 03:55:57 ozaki-r Exp $");
#ifdef _KERNEL_OPT
#include "opt_inet.h"
@@ -535,10 +535,10 @@ stf_checkaddr4(struct stf_softc *sc, con
(uint32_t)ntohl(sin.sin_addr.s_addr));
#endif
if (rt)
- rtfree(rt);
+ rt_unref(rt);
return -1;
}
- rtfree(rt);
+ rt_unref(rt);
}
return 0;
Index: src/sys/net/radix.c
diff -u src/sys/net/radix.c:1.46 src/sys/net/radix.c:1.47
--- src/sys/net/radix.c:1.46 Tue Nov 15 01:50:06 2016
+++ src/sys/net/radix.c Mon Dec 12 03:55:57 2016
@@ -1,4 +1,4 @@
-/* $NetBSD: radix.c,v 1.46 2016/11/15 01:50:06 ozaki-r Exp $ */
+/* $NetBSD: radix.c,v 1.47 2016/12/12 03:55:57 ozaki-r Exp $ */
/*
* Copyright (c) 1988, 1989, 1993
@@ -36,7 +36,7 @@
*/
#include <sys/cdefs.h>
-__KERNEL_RCSID(0, "$NetBSD: radix.c,v 1.46 2016/11/15 01:50:06 ozaki-r Exp $");
+__KERNEL_RCSID(0, "$NetBSD: radix.c,v 1.47 2016/12/12 03:55:57 ozaki-r Exp $");
#ifndef _NET_RADIX_H_
#include <sys/param.h>
@@ -1053,7 +1053,8 @@ rn_delayedinit(void **head, int off)
{
struct delayinit *di;
- KASSERT(radix_initialized == 0);
+ if (radix_initialized)
+ return;
di = kmem_alloc(sizeof(*di), KM_SLEEP);
di->head = head;
Index: src/sys/net/route.c
diff -u src/sys/net/route.c:1.182 src/sys/net/route.c:1.183
--- src/sys/net/route.c:1.182 Tue Nov 15 01:50:06 2016
+++ src/sys/net/route.c Mon Dec 12 03:55:57 2016
@@ -1,4 +1,4 @@
-/* $NetBSD: route.c,v 1.182 2016/11/15 01:50:06 ozaki-r Exp $ */
+/* $NetBSD: route.c,v 1.183 2016/12/12 03:55:57 ozaki-r Exp $ */
/*-
* Copyright (c) 1998, 2008 The NetBSD Foundation, Inc.
@@ -97,7 +97,7 @@
#endif
#include <sys/cdefs.h>
-__KERNEL_RCSID(0, "$NetBSD: route.c,v 1.182 2016/11/15 01:50:06 ozaki-r Exp $");
+__KERNEL_RCSID(0, "$NetBSD: route.c,v 1.183 2016/12/12 03:55:57 ozaki-r Exp $");
#include <sys/param.h>
#ifdef RTFLUSH_DEBUG
@@ -117,6 +117,9 @@ __KERNEL_RCSID(0, "$NetBSD: route.c,v 1.
#include <sys/kauth.h>
#include <sys/workqueue.h>
#include <sys/syslog.h>
+#include <sys/rwlock.h>
+#include <sys/mutex.h>
+#include <sys/cpu.h>
#include <net/if.h>
#include <net/if_dl.h>
@@ -131,6 +134,19 @@ __KERNEL_RCSID(0, "$NetBSD: route.c,v 1.
#define rtcache_debug() 0
#endif /* RTFLUSH_DEBUG */
+#ifdef RT_DEBUG
+#define RT_REFCNT_TRACE(rt) printf("%s:%d: rt=%p refcnt=%d\n", \
+ __func__, __LINE__, (rt), (rt)->rt_refcnt)
+#else
+#define RT_REFCNT_TRACE(rt) do {} while (0)
+#endif
+
+#ifdef DEBUG
+#define dlog(level, fmt, args...) log(level, fmt, ##args)
+#else
+#define dlog(level, fmt, args...) do {} while (0)
+#endif
+
struct rtstat rtstat;
static int rttrash; /* routes not in table but not freed */
@@ -147,6 +163,105 @@ static void rt_timer_queue_remove_all(st
static void rt_timer_remove_all(struct rtentry *);
static void rt_timer_timer(void *);
+/*
+ * Locking notes:
+ * - The routing table is protected by a global rwlock
+ * - API: RT_RLOCK and friends
+ * - rtcaches are protected by a global rwlock
+ * - API: RTCACHE_RLOCK and friends
+ * - References to a rtentry is managed by reference counting and psref
+ * - Reference couting is used for temporal reference when a rtentry
+ * is fetched from the routing table
+ * - psref is used for temporal reference when a rtentry is fetched
+ * from a rtcache
+ * - struct route (rtcache) has struct psref, so we cannot obtain
+ * a reference twice on the same struct route
+ * - Befere destroying or updating a rtentry, we have to wait for
+ * all references left (see below for details)
+ * - APIs
+ * - An obtained rtentry via rtalloc1 or rtrequest* must be
+ * unreferenced by rt_unref
+ * - An obtained rtentry via rtcache_* must be unreferenced by
+ * rtcache_unref
+ * - TODO: once we get a lockless routing table, we should use only
+ * psref for rtentries
+ * - rtentry destruction
+ * - A rtentry is destroyed (freed) only when we call rtrequest(RTM_DELETE)
+ * - If a caller of rtrequest grabs a reference of a rtentry, the caller
+ * has a responsibility to destroy the rtentry by itself by calling
+ * rt_free
+ * - If not, rtrequest itself does that
+ * - If rt_free is called in softint, the actual destruction routine is
+ * deferred to a workqueue
+ * - rtentry update
+ * - When updating a rtentry, RTF_UPDATING flag is set
+ * - If a rtentry is set RTF_UPDATING, fetching the rtentry from
+ * the routing table or a rtcache results in either of the following
+ * cases:
+ * - if the caller runs in softint, the caller fails to fetch
+ * - otherwise, the caller waits for the update completed and retries
+ * to fetch (probably succeed to fetch for the second time)
+ */
+
+/*
+ * Global locks for the routing table and rtcaches.
+ * Locking order: rtcache_lock => rt_lock
+ */
+static krwlock_t rt_lock __cacheline_aligned;
+#ifdef NET_MPSAFE
+#define RT_RLOCK() rw_enter(&rt_lock, RW_READER)
+#define RT_WLOCK() rw_enter(&rt_lock, RW_WRITER)
+#define RT_UNLOCK() rw_exit(&rt_lock)
+#define RT_LOCKED() rw_lock_held(&rt_lock)
+#define RT_ASSERT_WLOCK() KASSERT(rw_write_held(&rt_lock))
+#else
+#define RT_RLOCK() do {} while (0)
+#define RT_WLOCK() do {} while (0)
+#define RT_UNLOCK() do {} while (0)
+#define RT_LOCKED() false
+#define RT_ASSERT_WLOCK() do {} while (0)
+#endif
+
+static krwlock_t rtcache_lock __cacheline_aligned;
+#ifdef NET_MPSAFE
+#define RTCACHE_RLOCK() rw_enter(&rtcache_lock, RW_READER)
+#define RTCACHE_WLOCK() rw_enter(&rtcache_lock, RW_WRITER)
+#define RTCACHE_UNLOCK() rw_exit(&rtcache_lock)
+#define RTCACHE_ASSERT_WLOCK() KASSERT(rw_write_held(&rtcache_lock))
+#define RTCACHE_WLOCKED() rw_write_held(&rtcache_lock)
+#else
+#define RTCACHE_RLOCK() do {} while (0)
+#define RTCACHE_WLOCK() do {} while (0)
+#define RTCACHE_UNLOCK() do {} while (0)
+#define RTCACHE_ASSERT_WLOCK() do {} while (0)
+#define RTCACHE_WLOCKED() false
+#endif
+
+/*
+ * mutex and cv that are used to wait for references to a rtentry left
+ * before updating the rtentry.
+ */
+static struct {
+ kmutex_t lock;
+ kcondvar_t cv;
+ bool ongoing;
+ const struct lwp *lwp;
+} rt_update_global __cacheline_aligned;
+
+/*
+ * A workqueue and stuff that are used to defer the destruction routine
+ * of rtentries.
+ */
+static struct {
+ struct workqueue *wq;
+ struct work wk;
+ kmutex_t lock;
+ struct rtentry *queue[10];
+} rt_free_global __cacheline_aligned;
+
+/* psref for rtentry */
+static struct psref_class *rt_psref_class __read_mostly;
+
#ifdef RTFLUSH_DEBUG
static int _rtcache_debug = 0;
#endif /* RTFLUSH_DEBUG */
@@ -163,6 +278,23 @@ static void rtcache_clear(struct route *
static void rtcache_clear_rtentry(int, struct rtentry *);
static void rtcache_invalidate(struct dom_rtlist *);
+static void rt_ref(struct rtentry *);
+
+static struct rtentry *
+ rtalloc1_locked(const struct sockaddr *, int, bool);
+static struct rtentry *
+ rtcache_validate_locked(struct route *);
+static void rtcache_free_locked(struct route *);
+static int rtcache_setdst_locked(struct route *, const struct sockaddr *);
+
+static void rtcache_ref(struct rtentry *, struct route *);
+
+static void rt_update_wait(void);
+
+static bool rt_wait_ok(void);
+static void rt_wait_refcnt(const char *, struct rtentry *, int);
+static void rt_wait_psref(struct rtentry *);
+
#ifdef DDB
static void db_print_sa(const struct sockaddr *);
static void db_print_ifa(struct ifaddr *);
@@ -320,14 +452,28 @@ route_listener_cb(kauth_cred_t cred, kau
return result;
}
+static void rt_free_work(struct work *, void *);
+
void
rt_init(void)
{
+ int error;
#ifdef RTFLUSH_DEBUG
sysctl_net_rtcache_setup(NULL);
#endif
+ mutex_init(&rt_free_global.lock, MUTEX_DEFAULT, IPL_SOFTNET);
+ rt_psref_class = psref_class_create("rtentry", IPL_SOFTNET);
+
+ error = workqueue_create(&rt_free_global.wq, "rt_free",
+ rt_free_work, NULL, PRI_SOFTNET, IPL_SOFTNET, WQ_MPSAFE);
+ if (error)
+ panic("%s: workqueue_create failed (%d)\n", __func__, error);
+
+ mutex_init(&rt_update_global.lock, MUTEX_DEFAULT, IPL_SOFTNET);
+ cv_init(&rt_update_global.cv, "rt_update");
+
pool_init(&rtentry_pool, sizeof(struct rtentry), 0, 0, 0, "rtentpl",
NULL, IPL_SOFTNET);
pool_init(&rttimer_pool, sizeof(struct rttimer), 0, 0, 0, "rttmrpl",
@@ -351,7 +497,9 @@ rtflushall(int family)
if ((dom = pffinddomain(family)) == NULL)
return;
+ RTCACHE_WLOCK();
rtcache_invalidate(&dom->dom_rtcache);
+ RTCACHE_UNLOCK();
}
static void
@@ -359,6 +507,8 @@ rtcache(struct route *ro)
{
struct domain *dom;
+ RTCACHE_ASSERT_WLOCK();
+
rtcache_invariants(ro);
KASSERT(ro->_ro_rt != NULL);
KASSERT(ro->ro_invalid == false);
@@ -406,12 +556,13 @@ dump_rt(const struct rtentry *rt)
* will be incremented. The caller has to rtfree it by itself.
*/
struct rtentry *
-rtalloc1(const struct sockaddr *dst, int report)
+rtalloc1_locked(const struct sockaddr *dst, int report, bool wait_ok)
{
rtbl_t *rtbl;
struct rtentry *rt;
int s;
+retry:
s = splsoftnet();
rtbl = rt_gettable(dst->sa_family);
if (rtbl == NULL)
@@ -421,7 +572,34 @@ rtalloc1(const struct sockaddr *dst, int
if (rt == NULL)
goto miss;
- rt->rt_refcnt++;
+ if (!ISSET(rt->rt_flags, RTF_UP))
+ goto miss;
+
+ if (ISSET(rt->rt_flags, RTF_UPDATING) &&
+ /* XXX updater should be always able to acquire */
+ curlwp != rt_update_global.lwp) {
+ bool need_lock = false;
+ if (!wait_ok || !rt_wait_ok())
+ goto miss;
+ RT_UNLOCK();
+ splx(s);
+
+ /* XXX need more proper solution */
+ if (RTCACHE_WLOCKED()) {
+ RTCACHE_UNLOCK();
+ need_lock = true;
+ }
+
+ /* We can wait until the update is complete */
+ rt_update_wait();
+
+ if (need_lock)
+ RTCACHE_WLOCK();
+ goto retry;
+ }
+
+ rt_ref(rt);
+ RT_REFCNT_TRACE(rt);
splx(s);
return rt;
@@ -438,49 +616,211 @@ miss:
return NULL;
}
-#if defined(DEBUG) && !defined(NET_MPSAFE)
-/*
- * Check the following constraint for each rtcache:
- * if a rtcache holds a rtentry, the rtentry's refcnt is more than zero,
- * i.e., the rtentry should be referenced at least by the rtcache.
- */
+struct rtentry *
+rtalloc1(const struct sockaddr *dst, int report)
+{
+ struct rtentry *rt;
+
+ RT_RLOCK();
+ rt = rtalloc1_locked(dst, report, true);
+ RT_UNLOCK();
+
+ return rt;
+}
+
static void
-rtcache_check_rtrefcnt(int family)
+rt_ref(struct rtentry *rt)
{
- struct domain *dom = pffinddomain(family);
- struct route *ro;
- if (dom == NULL)
- return;
+ KASSERT(rt->rt_refcnt >= 0);
+ atomic_inc_uint(&rt->rt_refcnt);
+}
- LIST_FOREACH(ro, &dom->dom_rtcache, ro_rtcache_next)
- KDASSERT(ro->_ro_rt == NULL || ro->_ro_rt->rt_refcnt > 0);
+void
+rt_unref(struct rtentry *rt)
+{
+
+ KASSERT(rt != NULL);
+ KASSERTMSG(rt->rt_refcnt > 0, "refcnt=%d", rt->rt_refcnt);
+
+ atomic_dec_uint(&rt->rt_refcnt);
+ if (!ISSET(rt->rt_flags, RTF_UP) || ISSET(rt->rt_flags, RTF_UPDATING)) {
+ mutex_enter(&rt_free_global.lock);
+ cv_broadcast(&rt->rt_cv);
+ mutex_exit(&rt_free_global.lock);
+ }
+}
+
+static bool
+rt_wait_ok(void)
+{
+
+ KASSERT(!cpu_intr_p());
+ return !cpu_softintr_p();
+}
+
+void
+rt_wait_refcnt(const char *title, struct rtentry *rt, int cnt)
+{
+ mutex_enter(&rt_free_global.lock);
+ while (rt->rt_refcnt > cnt) {
+ dlog(LOG_DEBUG, "%s: %s waiting (refcnt=%d)\n",
+ __func__, title, rt->rt_refcnt);
+ cv_wait(&rt->rt_cv, &rt_free_global.lock);
+ dlog(LOG_DEBUG, "%s: %s waited (refcnt=%d)\n",
+ __func__, title, rt->rt_refcnt);
+ }
+ mutex_exit(&rt_free_global.lock);
}
-#endif
void
-rtfree(struct rtentry *rt)
+rt_wait_psref(struct rtentry *rt)
+{
+
+ psref_target_destroy(&rt->rt_psref, rt_psref_class);
+ psref_target_init(&rt->rt_psref, rt_psref_class);
+}
+
+static void
+_rt_free(struct rtentry *rt)
{
struct ifaddr *ifa;
- KASSERT(rt != NULL);
+ /*
+ * Need to avoid a deadlock on rt_wait_refcnt of update
+ * and a conflict on psref_target_destroy of update.
+ */
+ rt_update_wait();
+
+ RT_REFCNT_TRACE(rt);
+ KASSERTMSG(rt->rt_refcnt >= 0, "refcnt=%d", rt->rt_refcnt);
+ rt_wait_refcnt("free", rt, 0);
+ psref_target_destroy(&rt->rt_psref, rt_psref_class);
+
+ rt_assert_inactive(rt);
+ rttrash--;
+ ifa = rt->rt_ifa;
+ rt->rt_ifa = NULL;
+ ifafree(ifa);
+ rt->rt_ifp = NULL;
+ cv_destroy(&rt->rt_cv);
+ rt_destroy(rt);
+ pool_put(&rtentry_pool, rt);
+}
+
+static void
+rt_free_work(struct work *wk, void *arg)
+{
+ int i;
+ struct rtentry *rt;
+
+restart:
+ mutex_enter(&rt_free_global.lock);
+ for (i = 0; i < sizeof(rt_free_global.queue); i++) {
+ if (rt_free_global.queue[i] == NULL)
+ continue;
+ rt = rt_free_global.queue[i];
+ rt_free_global.queue[i] = NULL;
+ mutex_exit(&rt_free_global.lock);
+
+ atomic_dec_uint(&rt->rt_refcnt);
+ _rt_free(rt);
+ goto restart;
+ }
+ mutex_exit(&rt_free_global.lock);
+}
+
+void
+rt_free(struct rtentry *rt)
+{
+
KASSERT(rt->rt_refcnt > 0);
+ if (!rt_wait_ok()) {
+ int i;
+ mutex_enter(&rt_free_global.lock);
+ for (i = 0; i < sizeof(rt_free_global.queue); i++) {
+ if (rt_free_global.queue[i] == NULL) {
+ rt_free_global.queue[i] = rt;
+ break;
+ }
+ }
+ KASSERT(i < sizeof(rt_free_global.queue));
+ rt_ref(rt);
+ mutex_exit(&rt_free_global.lock);
+ workqueue_enqueue(rt_free_global.wq, &rt_free_global.wk, NULL);
+ } else {
+ atomic_dec_uint(&rt->rt_refcnt);
+ _rt_free(rt);
+ }
+}
- rt->rt_refcnt--;
-#if defined(DEBUG) && !defined(NET_MPSAFE)
- if (rt_getkey(rt) != NULL)
- rtcache_check_rtrefcnt(rt_getkey(rt)->sa_family);
-#endif
- if (rt->rt_refcnt == 0 && (rt->rt_flags & RTF_UP) == 0) {
- rt_assert_inactive(rt);
- rttrash--;
- ifa = rt->rt_ifa;
- rt->rt_ifa = NULL;
- ifafree(ifa);
- rt->rt_ifp = NULL;
- rt_destroy(rt);
- pool_put(&rtentry_pool, rt);
+static void
+rt_update_wait(void)
+{
+
+ mutex_enter(&rt_update_global.lock);
+ while (rt_update_global.ongoing) {
+ dlog(LOG_DEBUG, "%s: waiting lwp=%p\n", __func__, curlwp);
+ cv_wait(&rt_update_global.cv, &rt_update_global.lock);
+ dlog(LOG_DEBUG, "%s: waited lwp=%p\n", __func__, curlwp);
}
+ mutex_exit(&rt_update_global.lock);
+}
+
+int
+rt_update_prepare(struct rtentry *rt)
+{
+
+ dlog(LOG_DEBUG, "%s: updating rt=%p lwp=%p\n", __func__, rt, curlwp);
+
+ RTCACHE_WLOCK();
+ RT_WLOCK();
+ /* If the entry is being destroyed, don't proceed the update. */
+ if (!ISSET(rt->rt_flags, RTF_UP)) {
+ RT_UNLOCK();
+ RTCACHE_UNLOCK();
+ return -1;
+ }
+ rt->rt_flags |= RTF_UPDATING;
+ RT_UNLOCK();
+ RTCACHE_UNLOCK();
+
+ mutex_enter(&rt_update_global.lock);
+ while (rt_update_global.ongoing) {
+ dlog(LOG_DEBUG, "%s: waiting ongoing updating rt=%p lwp=%p\n",
+ __func__, rt, curlwp);
+ cv_wait(&rt_update_global.cv, &rt_update_global.lock);
+ dlog(LOG_DEBUG, "%s: waited ongoing updating rt=%p lwp=%p\n",
+ __func__, rt, curlwp);
+ }
+ rt_update_global.ongoing = true;
+ /* XXX need it to avoid rt_update_wait by updater itself. */
+ rt_update_global.lwp = curlwp;
+ mutex_exit(&rt_update_global.lock);
+
+ rt_wait_refcnt("update", rt, 1);
+ rt_wait_psref(rt);
+
+ return 0;
+}
+
+void
+rt_update_finish(struct rtentry *rt)
+{
+
+ RTCACHE_WLOCK();
+ RT_WLOCK();
+ rt->rt_flags &= ~RTF_UPDATING;
+ RT_UNLOCK();
+ RTCACHE_UNLOCK();
+
+ mutex_enter(&rt_update_global.lock);
+ rt_update_global.ongoing = false;
+ rt_update_global.lwp = NULL;
+ cv_broadcast(&rt_update_global.cv);
+ mutex_exit(&rt_update_global.lock);
+
+ dlog(LOG_DEBUG, "%s: updated rt=%p lwp=%p\n", __func__, rt, curlwp);
}
/*
@@ -549,7 +889,7 @@ rtredirect(const struct sockaddr *dst, c
*/
create:
if (rt != NULL)
- rtfree(rt);
+ rt_unref(rt);
flags |= RTF_GATEWAY | RTF_DYNAMIC;
memset(&info, 0, sizeof(info));
info.rti_info[RTAX_DST] = dst;
@@ -567,6 +907,10 @@ rtredirect(const struct sockaddr *dst, c
* Smash the current notion of the gateway to
* this destination. Should check about netmask!!!
*/
+ /*
+ * FIXME NOMPAFE: the rtentry is updated with the existence
+ * of refeferences of it.
+ */
error = rt_setgate(rt, gateway);
if (error == 0) {
rt->rt_flags |= RTF_MODIFIED;
@@ -581,7 +925,7 @@ done:
if (rtp != NULL && !error)
*rtp = rt;
else
- rtfree(rt);
+ rt_unref(rt);
}
out:
if (error)
@@ -622,8 +966,6 @@ rtdeletemsg(struct rtentry *rt)
rt_missmsg(RTM_DELETE, &info, info.rti_flags, error);
- if (error == 0)
- rtfree(retrt);
return error;
}
@@ -674,7 +1016,7 @@ ifa_ifwithroute_psref(int flags, const s
if (ifa != NULL)
ifa_acquire(ifa, psref);
pserialize_read_exit(s);
- rtfree(rt);
+ rt_unref(rt);
if (ifa == NULL)
return NULL;
}
@@ -733,7 +1075,10 @@ rtrequest_newmsg(const int req, const st
KASSERT(ret_nrt != NULL);
rt_newmsg(req, ret_nrt); /* tell user process */
- rtfree(ret_nrt);
+ if (req == RTM_DELETE)
+ rt_free(ret_nrt);
+ else
+ rt_unref(ret_nrt);
return 0;
}
@@ -824,8 +1169,11 @@ rtrequest1(int req, struct rt_addrinfo *
int bound = 0;
struct ifnet *ifp = NULL;
bool need_to_release_ifa = true;
+ bool need_unlock = true;
#define senderr(x) { error = x ; goto bad; }
+ RT_WLOCK();
+
bound = curlwp_bind();
if ((rtbl = rt_gettable(dst->sa_family)) == NULL)
senderr(ESRCH);
@@ -856,16 +1204,21 @@ rtrequest1(int req, struct rt_addrinfo *
ifa = NULL;
}
rttrash++;
- rt_timer_remove_all(rt);
if (ret_nrt) {
*ret_nrt = rt;
- rt->rt_refcnt++;
- } else if (rt->rt_refcnt <= 0) {
- /* Adjust the refcount */
- rt->rt_refcnt++;
- rtfree(rt);
+ rt_ref(rt);
+ RT_REFCNT_TRACE(rt);
}
+ RT_UNLOCK();
+ need_unlock = false;
+ rt_timer_remove_all(rt);
rtcache_clear_rtentry(dst->sa_family, rt);
+ if (ret_nrt == NULL) {
+ /* Adjust the refcount */
+ rt_ref(rt);
+ RT_REFCNT_TRACE(rt);
+ rt_free(rt);
+ }
break;
case RTM_ADD:
@@ -920,12 +1273,15 @@ rtrequest1(int req, struct rt_addrinfo *
} else
rt->rt_ifp = ifa->ifa_ifp;
pserialize_read_exit(ss);
+ cv_init(&rt->rt_cv, "rtentry");
+ psref_target_init(&rt->rt_psref, rt_psref_class);
RT_DPRINTF("rt->_rt_key = %p\n", (void *)rt->_rt_key);
rc = rt_addaddr(rtbl, rt, netmask);
RT_DPRINTF("rt->_rt_key = %p\n", (void *)rt->_rt_key);
if (rc != 0) {
ifafree(ifa); /* for rt_set_ifa above */
+ cv_destroy(&rt->rt_cv);
rt_destroy(rt);
pool_put(&rtentry_pool, rt);
senderr(rc);
@@ -941,8 +1297,11 @@ rtrequest1(int req, struct rt_addrinfo *
RT_DPRINTF("rt->_rt_key = %p\n", (void *)rt->_rt_key);
if (ret_nrt) {
*ret_nrt = rt;
- rt->rt_refcnt++;
+ rt_ref(rt);
+ RT_REFCNT_TRACE(rt);
}
+ RT_UNLOCK();
+ need_unlock = false;
rtflushall(dst->sa_family);
break;
case RTM_GET:
@@ -955,7 +1314,8 @@ rtrequest1(int req, struct rt_addrinfo *
senderr(ESRCH);
if (ret_nrt != NULL) {
*ret_nrt = rt;
- rt->rt_refcnt++;
+ rt_ref(rt);
+ RT_REFCNT_TRACE(rt);
}
break;
}
@@ -964,6 +1324,8 @@ bad:
ifa_release(ifa, &psref_ifa);
if_put(ifp, &psref_ifp);
curlwp_bindx(bound);
+ if (need_unlock)
+ RT_UNLOCK();
splx(s);
return error;
}
@@ -989,7 +1351,13 @@ rt_setgate(struct rtentry *rt, const str
RT_DPRINTF("rt->_rt_key = %p\n", (void *)rt->_rt_key);
if (rt->rt_flags & RTF_GATEWAY) {
- struct rtentry *gwrt = rtalloc1(gate, 1);
+ struct rtentry *gwrt;
+
+ /* XXX we cannot call rtalloc1 if holding the rt lock */
+ if (RT_LOCKED())
+ gwrt = rtalloc1_locked(gate, 1, false);
+ else
+ gwrt = rtalloc1(gate, 1);
/*
* If we switched gateways, grab the MTU from the new
* gateway route if the current MTU, if the current MTU is
@@ -1005,7 +1373,7 @@ rt_setgate(struct rtentry *rt, const str
rt->rt_rmx.rmx_mtu > gwrt->rt_rmx.rmx_mtu) {
rt->rt_rmx.rmx_mtu = gwrt->rt_rmx.rmx_mtu;
}
- rtfree(gwrt);
+ rt_unref(gwrt);
}
}
KASSERT(rt->_rt_key != NULL);
@@ -1076,11 +1444,11 @@ rtinit(struct ifaddr *ifa, int cmd, int
}
if ((rt = rtalloc1(dst, 0)) != NULL) {
if (rt->rt_ifa != ifa) {
- rtfree(rt);
+ rt_unref(rt);
return (flags & RTF_HOST) ? EHOSTUNREACH
: ENETUNREACH;
}
- rtfree(rt);
+ rt_unref(rt);
}
}
memset(&info, 0, sizeof(info));
@@ -1103,16 +1471,29 @@ rtinit(struct ifaddr *ifa, int cmd, int
return error;
rt = nrt;
+ RT_REFCNT_TRACE(rt);
switch (cmd) {
case RTM_DELETE:
rt_newmsg(cmd, rt);
+ rt_free(rt);
break;
case RTM_LLINFO_UPD:
if (cmd == RTM_LLINFO_UPD && ifa->ifa_rtrequest != NULL)
ifa->ifa_rtrequest(RTM_LLINFO_UPD, rt, &info);
rt_newmsg(RTM_CHANGE, rt);
+ rt_unref(rt);
break;
case RTM_ADD:
+ /*
+ * FIXME NOMPAFE: the rtentry is updated with the existence
+ * of refeferences of it.
+ */
+ /*
+ * XXX it looks just reverting rt_ifa replaced by ifa_rtrequest
+ * called via rtrequest1. Can we just prevent the replacement
+ * somehow and remove the following code? And also doesn't
+ * calling ifa_rtrequest(RTM_ADD) replace rt_ifa again?
+ */
if (rt->rt_ifa != ifa) {
printf("rtinit: wrong ifa (%p) was (%p)\n", ifa,
rt->rt_ifa);
@@ -1126,9 +1507,10 @@ rtinit(struct ifaddr *ifa, int cmd, int
ifa->ifa_rtrequest(RTM_ADD, rt, &info);
}
rt_newmsg(cmd, rt);
+ rt_unref(rt);
+ RT_REFCNT_TRACE(rt);
break;
}
- rtfree(rt);
return error;
}
@@ -1171,14 +1553,15 @@ rt_ifa_addlocal(struct ifaddr *ifa)
#ifdef RT_DEBUG
dump_rt(nrt);
#endif
- rtfree(nrt);
+ rt_unref(nrt);
+ RT_REFCNT_TRACE(nrt);
}
} else {
e = 0;
rt_newaddrmsg(RTM_NEWADDR, ifa, 0, NULL);
}
if (rt != NULL)
- rtfree(rt);
+ rt_unref(rt);
return e;
}
@@ -1212,6 +1595,11 @@ rt_ifa_remlocal(struct ifaddr *ifa, stru
*/
if (alt_ifa == NULL) {
e = rtdeletemsg(rt);
+ if (e == 0) {
+ rt_unref(rt);
+ rt_free(rt);
+ rt = NULL;
+ }
rt_newaddrmsg(RTM_DELADDR, ifa, 0, NULL);
} else {
rt_replace_ifa(rt, alt_ifa);
@@ -1220,7 +1608,7 @@ rt_ifa_remlocal(struct ifaddr *ifa, stru
} else
rt_newaddrmsg(RTM_DELADDR, ifa, 0, NULL);
if (rt != NULL)
- rtfree(rt);
+ rt_unref(rt);
return e;
}
@@ -1252,6 +1640,10 @@ rt_timer_init(void)
assert(rt_init_done == 0);
+ /* XXX should be in rt_init */
+ rw_init(&rt_lock);
+ rw_init(&rtcache_lock);
+
LIST_INIT(&rttimer_queue_head);
callout_init(&rt_timer_ch, CALLOUT_MPSAFE);
error = workqueue_create(&rt_timer_wq, "rt_timer",
@@ -1277,7 +1669,9 @@ rt_timer_queue_create(u_int timeout)
rtq->rtq_timeout = timeout;
TAILQ_INIT(&rtq->rtq_head);
+ RT_WLOCK();
LIST_INSERT_HEAD(&rttimer_queue_head, rtq, rtq_link);
+ RT_UNLOCK();
return rtq;
}
@@ -1294,12 +1688,17 @@ rt_timer_queue_remove_all(struct rttimer
{
struct rttimer *r;
+ RT_ASSERT_WLOCK();
+
while ((r = TAILQ_FIRST(&rtq->rtq_head)) != NULL) {
LIST_REMOVE(r, rtt_link);
TAILQ_REMOVE(&rtq->rtq_head, r, rtt_next);
+ rt_ref(r->rtt_rt); /* XXX */
+ RT_REFCNT_TRACE(r->rtt_rt);
+ RT_UNLOCK();
(*r->rtt_func)(r->rtt_rt, r);
- rtfree(r->rtt_rt);
pool_put(&rttimer_pool, r);
+ RT_WLOCK();
if (rtq->rtq_count > 0)
rtq->rtq_count--;
else
@@ -1312,9 +1711,10 @@ void
rt_timer_queue_destroy(struct rttimer_queue *rtq)
{
+ RT_WLOCK();
rt_timer_queue_remove_all(rtq);
-
LIST_REMOVE(rtq, rtq_link);
+ RT_UNLOCK();
/*
* Caller is responsible for freeing the rttimer_queue structure.
@@ -1332,6 +1732,7 @@ rt_timer_remove_all(struct rtentry *rt)
{
struct rttimer *r;
+ RT_WLOCK();
while ((r = LIST_FIRST(&rt->rt_timer)) != NULL) {
LIST_REMOVE(r, rtt_link);
TAILQ_REMOVE(&r->rtt_queue->rtq_head, r, rtt_next);
@@ -1340,8 +1741,8 @@ rt_timer_remove_all(struct rtentry *rt)
else
printf("rt_timer_remove_all: rtq_count reached 0\n");
pool_put(&rttimer_pool, r);
- rt->rt_refcnt--; /* XXX */
}
+ RT_UNLOCK();
}
int
@@ -1352,6 +1753,7 @@ rt_timer_add(struct rtentry *rt,
struct rttimer *r;
KASSERT(func != NULL);
+ RT_WLOCK();
/*
* If there's already a timer with this action, destroy it before
* we add a new one.
@@ -1367,16 +1769,16 @@ rt_timer_add(struct rtentry *rt,
r->rtt_queue->rtq_count--;
else
printf("rt_timer_add: rtq_count reached 0\n");
- rtfree(r->rtt_rt);
} else {
r = pool_get(&rttimer_pool, PR_NOWAIT);
- if (r == NULL)
+ if (r == NULL) {
+ RT_UNLOCK();
return ENOBUFS;
+ }
}
memset(r, 0, sizeof(*r));
- rt->rt_refcnt++;
r->rtt_rt = rt;
r->rtt_time = time_uptime;
r->rtt_func = func;
@@ -1385,6 +1787,8 @@ rt_timer_add(struct rtentry *rt,
TAILQ_INSERT_TAIL(&queue->rtq_head, r, rtt_next);
r->rtt_queue->rtq_count++;
+ RT_UNLOCK();
+
return 0;
}
@@ -1393,24 +1797,26 @@ rt_timer_work(struct work *wk, void *arg
{
struct rttimer_queue *rtq;
struct rttimer *r;
- int s;
- s = splsoftnet();
+ RT_WLOCK();
LIST_FOREACH(rtq, &rttimer_queue_head, rtq_link) {
while ((r = TAILQ_FIRST(&rtq->rtq_head)) != NULL &&
(r->rtt_time + rtq->rtq_timeout) < time_uptime) {
LIST_REMOVE(r, rtt_link);
TAILQ_REMOVE(&rtq->rtq_head, r, rtt_next);
+ rt_ref(r->rtt_rt); /* XXX */
+ RT_REFCNT_TRACE(r->rtt_rt);
+ RT_UNLOCK();
(*r->rtt_func)(r->rtt_rt, r);
- rtfree(r->rtt_rt);
pool_put(&rttimer_pool, r);
+ RT_WLOCK();
if (rtq->rtq_count > 0)
rtq->rtq_count--;
else
printf("rt_timer_timer: rtq_count reached 0\n");
}
}
- splx(s);
+ RT_UNLOCK();
callout_reset(&rt_timer_ch, hz, rt_timer_timer, NULL);
}
@@ -1425,14 +1831,24 @@ rt_timer_timer(void *arg)
static struct rtentry *
_rtcache_init(struct route *ro, int flag)
{
+ struct rtentry *rt;
+
rtcache_invariants(ro);
KASSERT(ro->_ro_rt == NULL);
+ RTCACHE_ASSERT_WLOCK();
if (rtcache_getdst(ro) == NULL)
return NULL;
ro->ro_invalid = false;
- if ((ro->_ro_rt = rtalloc1(rtcache_getdst(ro), flag)) != NULL)
+ rt = rtalloc1(rtcache_getdst(ro), flag);
+ if (rt != NULL && ISSET(rt->rt_flags, RTF_UP)) {
+ ro->_ro_rt = rt;
+ KASSERT(!ISSET(rt->rt_flags, RTF_UPDATING));
+ rtcache_ref(rt, ro);
+ rt_unref(rt);
rtcache(ro);
+ } else if (rt != NULL)
+ rt_unref(rt);
rtcache_invariants(ro);
return ro->_ro_rt;
@@ -1441,51 +1857,161 @@ _rtcache_init(struct route *ro, int flag
struct rtentry *
rtcache_init(struct route *ro)
{
- return _rtcache_init(ro, 1);
+ struct rtentry *rt;
+ RTCACHE_WLOCK();
+ rt = _rtcache_init(ro, 1);
+ RTCACHE_UNLOCK();
+ return rt;
}
struct rtentry *
rtcache_init_noclone(struct route *ro)
{
- return _rtcache_init(ro, 0);
+ struct rtentry *rt;
+ RTCACHE_WLOCK();
+ rt = _rtcache_init(ro, 0);
+ RTCACHE_UNLOCK();
+ return rt;
}
struct rtentry *
rtcache_update(struct route *ro, int clone)
{
+ struct rtentry *rt;
+ RTCACHE_WLOCK();
rtcache_clear(ro);
- return _rtcache_init(ro, clone);
+ rt = _rtcache_init(ro, clone);
+ RTCACHE_UNLOCK();
+ return rt;
}
void
-rtcache_copy(struct route *new_ro, const struct route *old_ro)
+rtcache_copy(struct route *new_ro, struct route *old_ro)
{
struct rtentry *rt;
+ int ret;
KASSERT(new_ro != old_ro);
rtcache_invariants(new_ro);
rtcache_invariants(old_ro);
- if ((rt = rtcache_validate(old_ro)) != NULL)
- rt->rt_refcnt++;
+ rt = rtcache_validate(old_ro);
- if (rtcache_getdst(old_ro) == NULL ||
- rtcache_setdst(new_ro, rtcache_getdst(old_ro)) != 0)
- return;
+ if (rtcache_getdst(old_ro) == NULL)
+ goto out;
+ ret = rtcache_setdst(new_ro, rtcache_getdst(old_ro));
+ if (ret != 0)
+ goto out;
+ RTCACHE_WLOCK();
new_ro->ro_invalid = false;
if ((new_ro->_ro_rt = rt) != NULL)
rtcache(new_ro);
rtcache_invariants(new_ro);
+ RTCACHE_UNLOCK();
+out:
+ rtcache_unref(rt, old_ro);
+ return;
}
static struct dom_rtlist invalid_routes = LIST_HEAD_INITIALIZER(dom_rtlist);
+#ifdef RT_DEBUG
+static void
+rtcache_trace(const char *func, struct rtentry *rt, struct route *ro)
+{
+ char dst[64];
+
+ sockaddr_format(ro->ro_sa, dst, 64);
+ printf("trace: %s:\tdst=%s cpu=%d lwp=%p psref=%p target=%p\n", func, dst,
+ cpu_index(curcpu()), curlwp, &ro->ro_psref, &rt->rt_psref);
+}
+#define RTCACHE_PSREF_TRACE(rt, ro) rtcache_trace(__func__, (rt), (ro))
+#else
+#define RTCACHE_PSREF_TRACE(rt, ro) do {} while (0)
+#endif
+
+static void
+rtcache_ref(struct rtentry *rt, struct route *ro)
+{
+
+ KASSERT(rt != NULL);
+
+#ifdef NET_MPSAFE
+ RTCACHE_PSREF_TRACE(rt, ro);
+ ro->ro_bound = curlwp_bind();
+ psref_acquire(&ro->ro_psref, &rt->rt_psref, rt_psref_class);
+#endif
+}
+
+void
+rtcache_unref(struct rtentry *rt, struct route *ro)
+{
+
+ if (rt == NULL)
+ return;
+
+#ifdef NET_MPSAFE
+ psref_release(&ro->ro_psref, &rt->rt_psref, rt_psref_class);
+ curlwp_bindx(ro->ro_bound);
+ RTCACHE_PSREF_TRACE(rt, ro);
+#endif
+}
+
+static struct rtentry *
+rtcache_validate_locked(struct route *ro)
+{
+ struct rtentry *rt = NULL;
+
+retry:
+ rt = ro->_ro_rt;
+ rtcache_invariants(ro);
+
+ if (ro->ro_invalid) {
+ rt = NULL;
+ goto out;
+ }
+
+ RT_RLOCK();
+ if (rt != NULL && (rt->rt_flags & RTF_UP) != 0 && rt->rt_ifp != NULL) {
+ if (ISSET(rt->rt_flags, RTF_UPDATING)) {
+ if (rt_wait_ok()) {
+ RT_UNLOCK();
+ RTCACHE_UNLOCK();
+ /* We can wait until the update is complete */
+ rt_update_wait();
+ RTCACHE_RLOCK();
+ goto retry;
+ } else {
+ rt = NULL;
+ }
+ } else
+ rtcache_ref(rt, ro);
+ } else
+ rt = NULL;
+ RT_UNLOCK();
+out:
+ return rt;
+}
+
+struct rtentry *
+rtcache_validate(struct route *ro)
+{
+ struct rtentry *rt;
+
+ RTCACHE_RLOCK();
+ rt = rtcache_validate_locked(ro);
+ RTCACHE_UNLOCK();
+ return rt;
+}
+
static void
rtcache_invalidate(struct dom_rtlist *rtlist)
{
struct route *ro;
+ RTCACHE_ASSERT_WLOCK();
+
while ((ro = LIST_FIRST(rtlist)) != NULL) {
rtcache_invariants(ro);
KASSERT(ro->_ro_rt != NULL);
@@ -1505,66 +2031,84 @@ rtcache_clear_rtentry(int family, struct
if ((dom = pffinddomain(family)) == NULL)
return;
+ RTCACHE_WLOCK();
LIST_FOREACH_SAFE(ro, &dom->dom_rtcache, ro_rtcache_next, nro) {
if (ro->_ro_rt == rt)
rtcache_clear(ro);
}
+ RTCACHE_UNLOCK();
}
static void
rtcache_clear(struct route *ro)
{
+
+ RTCACHE_ASSERT_WLOCK();
+
rtcache_invariants(ro);
if (ro->_ro_rt == NULL)
return;
LIST_REMOVE(ro, ro_rtcache_next);
- rtfree(ro->_ro_rt);
ro->_ro_rt = NULL;
ro->ro_invalid = false;
rtcache_invariants(ro);
}
struct rtentry *
-rtcache_lookup2(struct route *ro, const struct sockaddr *dst, int clone,
- int *hitp)
+rtcache_lookup2(struct route *ro, const struct sockaddr *dst,
+ int clone, int *hitp)
{
const struct sockaddr *odst;
struct rtentry *rt = NULL;
+ RTCACHE_RLOCK();
odst = rtcache_getdst(ro);
- if (odst == NULL)
+ if (odst == NULL) {
+ RTCACHE_UNLOCK();
+ RTCACHE_WLOCK();
goto miss;
+ }
if (sockaddr_cmp(odst, dst) != 0) {
- rtcache_free(ro);
+ RTCACHE_UNLOCK();
+ RTCACHE_WLOCK();
+ rtcache_free_locked(ro);
goto miss;
}
- rt = rtcache_validate(ro);
+ rt = rtcache_validate_locked(ro);
if (rt == NULL) {
+ RTCACHE_UNLOCK();
+ RTCACHE_WLOCK();
rtcache_clear(ro);
goto miss;
}
- *hitp = 1;
rtcache_invariants(ro);
+ RTCACHE_UNLOCK();
+ if (hitp != NULL)
+ *hitp = 1;
return rt;
miss:
- *hitp = 0;
- if (rtcache_setdst(ro, dst) == 0)
+ if (hitp != NULL)
+ *hitp = 0;
+ if (rtcache_setdst_locked(ro, dst) == 0)
rt = _rtcache_init(ro, clone);
rtcache_invariants(ro);
+ RTCACHE_UNLOCK();
return rt;
}
-void
-rtcache_free(struct route *ro)
+static void
+rtcache_free_locked(struct route *ro)
{
+
+ RTCACHE_ASSERT_WLOCK();
rtcache_clear(ro);
if (ro->ro_sa != NULL) {
sockaddr_free(ro->ro_sa);
@@ -1573,11 +2117,22 @@ rtcache_free(struct route *ro)
rtcache_invariants(ro);
}
-int
-rtcache_setdst(struct route *ro, const struct sockaddr *sa)
+void
+rtcache_free(struct route *ro)
+{
+
+ RTCACHE_WLOCK();
+ rtcache_free_locked(ro);
+ RTCACHE_UNLOCK();
+}
+
+static int
+rtcache_setdst_locked(struct route *ro, const struct sockaddr *sa)
{
KASSERT(sa != NULL);
+ RTCACHE_ASSERT_WLOCK();
+
rtcache_invariants(ro);
if (ro->ro_sa != NULL) {
if (ro->ro_sa->sa_family == sa->sa_family) {
@@ -1587,7 +2142,7 @@ rtcache_setdst(struct route *ro, const s
return 0;
}
/* free ro_sa, wrong family */
- rtcache_free(ro);
+ rtcache_free_locked(ro);
}
KASSERT(ro->_ro_rt == NULL);
@@ -1600,6 +2155,18 @@ rtcache_setdst(struct route *ro, const s
return 0;
}
+int
+rtcache_setdst(struct route *ro, const struct sockaddr *sa)
+{
+ int error;
+
+ RTCACHE_WLOCK();
+ error = rtcache_setdst_locked(ro, sa);
+ RTCACHE_UNLOCK();
+
+ return error;
+}
+
const struct sockaddr *
rt_settag(struct rtentry *rt, const struct sockaddr *tag)
{
@@ -1645,14 +2212,17 @@ rt_delete_matched_entries(sa_family_t fa
int error;
struct rtentry *rt, *retrt = NULL;
+ RT_RLOCK();
s = splsoftnet();
rt = rtbl_search_matched_entry(family, f, v);
if (rt == NULL) {
splx(s);
+ RT_UNLOCK();
return;
}
rt->rt_refcnt++;
splx(s);
+ RT_UNLOCK();
error = rtrequest(RTM_DELETE, rt_getkey(rt), rt->rt_gateway,
rt_mask(rt), rt->rt_flags, &retrt);
@@ -1660,11 +2230,11 @@ rt_delete_matched_entries(sa_family_t fa
KASSERT(retrt == rt);
KASSERT((retrt->rt_flags & RTF_UP) == 0);
retrt->rt_ifp = NULL;
- rtfree(rt);
- rtfree(retrt);
+ rt_unref(rt);
+ rt_free(retrt);
} else if (error == ESRCH) {
/* Someone deleted the entry already. */
- rtfree(rt);
+ rt_unref(rt);
} else {
log(LOG_ERR, "%s: unable to delete rtentry @ %p, "
"error = %d\n", rt->rt_ifp->if_xname, rt, error);
@@ -1673,6 +2243,18 @@ rt_delete_matched_entries(sa_family_t fa
}
}
+int
+rt_walktree(sa_family_t family, int (*f)(struct rtentry *, void *), void *v)
+{
+ int error;
+
+ RT_RLOCK();
+ error = rtbl_walktree(family, f, v);
+ RT_UNLOCK();
+
+ return error;
+}
+
#ifdef DDB
#include <machine/db_machdep.h>
Index: src/sys/net/route.h
diff -u src/sys/net/route.h:1.108 src/sys/net/route.h:1.109
--- src/sys/net/route.h:1.108 Thu Dec 8 05:16:33 2016
+++ src/sys/net/route.h Mon Dec 12 03:55:57 2016
@@ -1,4 +1,4 @@
-/* $NetBSD: route.h,v 1.108 2016/12/08 05:16:33 ozaki-r Exp $ */
+/* $NetBSD: route.h,v 1.109 2016/12/12 03:55:57 ozaki-r Exp $ */
/*
* Copyright (c) 1980, 1986, 1993
@@ -38,6 +38,12 @@
#include <sys/socket.h>
#include <sys/types.h>
#include <net/if.h>
+#ifdef _KERNEL
+#include <sys/rwlock.h>
+#include <sys/condvar.h>
+#include <sys/pserialize.h>
+#include <sys/psref.h>
+#endif
#if !(defined(_KERNEL) || defined(_STANDALONE))
#include <stdbool.h>
@@ -60,6 +66,10 @@ struct route {
struct sockaddr *ro_sa;
LIST_ENTRY(route) ro_rtcache_next;
bool ro_invalid;
+#ifdef _KERNEL
+ struct psref ro_psref;
+ int ro_bound;
+#endif
};
/*
@@ -115,6 +125,10 @@ struct rtentry {
struct rtentry *rt_parent; /* parent of cloned route */
struct sockaddr *_rt_key;
struct sockaddr *rt_tag; /* route tagging info */
+#ifdef _KERNEL
+ kcondvar_t rt_cv;
+ struct psref_target rt_psref;
+#endif
};
static inline const struct sockaddr *
@@ -159,6 +173,7 @@ struct ortentry {
#define RTF_ANNOUNCE 0x20000 /* announce new ARP or NDP entry */
#define RTF_LOCAL 0x40000 /* route represents a local address */
#define RTF_BROADCAST 0x80000 /* route represents a bcast address */
+#define RTF_UPDATING 0x100000 /* route is updating */
/*
* Routing statistics.
@@ -376,10 +391,15 @@ struct rttimer_queue *
rt_timer_queue_create(u_int);
void rt_timer_queue_destroy(struct rttimer_queue *);
+void rt_free(struct rtentry *);
+void rt_unref(struct rtentry *);
+
+int rt_update_prepare(struct rtentry *);
+void rt_update_finish(struct rtentry *);
+
void rt_newmsg(const int, const struct rtentry *);
struct rtentry *
rtalloc1(const struct sockaddr *, int);
-void rtfree(struct rtentry *);
int rtinit(struct ifaddr *, int, int);
void rtredirect(const struct sockaddr *, const struct sockaddr *,
const struct sockaddr *, int, const struct sockaddr *,
@@ -410,6 +430,7 @@ struct sockaddr *
int rt_check_reject_route(const struct rtentry *, const struct ifnet *);
void rt_delete_matched_entries(sa_family_t,
int (*)(struct rtentry *, void *), void *);
+int rt_walktree(sa_family_t, int (*)(struct rtentry *, void *), void *);
static inline void
rt_assert_referenced(const struct rtentry *rt)
@@ -418,7 +439,7 @@ rt_assert_referenced(const struct rtentr
KASSERT(rt->rt_refcnt > 0);
}
-void rtcache_copy(struct route *, const struct route *);
+void rtcache_copy(struct route *, struct route *);
void rtcache_free(struct route *);
struct rtentry *
rtcache_init(struct route *);
@@ -436,7 +457,6 @@ rtcache_invariants(const struct route *r
{
KASSERT(ro->ro_sa != NULL || ro->_ro_rt == NULL);
KASSERT(!ro->ro_invalid || ro->_ro_rt != NULL);
- KASSERT(ro->_ro_rt == NULL || ro->_ro_rt->rt_refcnt > 0);
}
static inline struct rtentry *
@@ -456,36 +476,15 @@ rtcache_lookup(struct route *ro, const s
static inline const struct sockaddr *
rtcache_getdst(const struct route *ro)
{
- rtcache_invariants(ro);
- return ro->ro_sa;
-}
-
-/* If the cache is not empty, and the cached route is still present
- * in the routing table, return the cached route. Otherwise, return
- * NULL.
- */
-static inline struct rtentry *
-rtcache_validate(const struct route *ro)
-{
- struct rtentry *rt = ro->_ro_rt;
rtcache_invariants(ro);
-
- if (ro->ro_invalid)
- return NULL;
-
- if (rt != NULL && (rt->rt_flags & RTF_UP) != 0 && rt->rt_ifp != NULL)
- return rt;
- return NULL;
-
+ return ro->ro_sa;
}
-static inline void
-rtcache_unref(struct rtentry *rt, struct route *ro)
-{
+struct rtentry *
+ rtcache_validate(struct route *);
- /* Will do something useful in the future. */
-}
+void rtcache_unref(struct rtentry *, struct route *);
/* rtsock */
void rt_ieee80211msg(struct ifnet *, int, void *, size_t);
@@ -512,7 +511,7 @@ struct rtentry *
struct rtentry *
rt_matchaddr(rtbl_t *, const struct sockaddr *);
int rt_refines(const struct sockaddr *, const struct sockaddr *);
-int rt_walktree(sa_family_t, int (*)(struct rtentry *, void *), void *);
+int rtbl_walktree(sa_family_t, int (*)(struct rtentry *, void *), void *);
struct rtentry *
rtbl_search_matched_entry(sa_family_t,
int (*)(struct rtentry *, void *), void *);
Index: src/sys/net/rtbl.c
diff -u src/sys/net/rtbl.c:1.4 src/sys/net/rtbl.c:1.5
--- src/sys/net/rtbl.c:1.4 Tue Nov 15 01:50:06 2016
+++ src/sys/net/rtbl.c Mon Dec 12 03:55:57 2016
@@ -1,4 +1,4 @@
-/* $NetBSD: rtbl.c,v 1.4 2016/11/15 01:50:06 ozaki-r Exp $ */
+/* $NetBSD: rtbl.c,v 1.5 2016/12/12 03:55:57 ozaki-r Exp $ */
/*-
* Copyright (c) 1998, 2008, 2011 The NetBSD Foundation, Inc.
@@ -95,7 +95,7 @@
#endif /* _KERNEL && _KERNEL_OPT */
#include <sys/cdefs.h>
-__KERNEL_RCSID(0, "$NetBSD: rtbl.c,v 1.4 2016/11/15 01:50:06 ozaki-r Exp $");
+__KERNEL_RCSID(0, "$NetBSD: rtbl.c,v 1.5 2016/12/12 03:55:57 ozaki-r Exp $");
#include <sys/param.h>
#include <sys/kmem.h>
@@ -190,7 +190,7 @@ rt_walktree_visitor(struct radix_node *r
}
int
-rt_walktree(sa_family_t family, int (*f)(struct rtentry *, void *), void *v)
+rtbl_walktree(sa_family_t family, int (*f)(struct rtentry *, void *), void *v)
{
rtbl_t *t = rt_tables[family];
struct rtwalk rw;
Index: src/sys/net/rtsock.c
diff -u src/sys/net/rtsock.c:1.198 src/sys/net/rtsock.c:1.199
--- src/sys/net/rtsock.c:1.198 Wed Oct 26 06:49:10 2016
+++ src/sys/net/rtsock.c Mon Dec 12 03:55:57 2016
@@ -1,4 +1,4 @@
-/* $NetBSD: rtsock.c,v 1.198 2016/10/26 06:49:10 ozaki-r Exp $ */
+/* $NetBSD: rtsock.c,v 1.199 2016/12/12 03:55:57 ozaki-r Exp $ */
/*
* Copyright (C) 1995, 1996, 1997, and 1998 WIDE Project.
@@ -61,7 +61,7 @@
*/
#include <sys/cdefs.h>
-__KERNEL_RCSID(0, "$NetBSD: rtsock.c,v 1.198 2016/10/26 06:49:10 ozaki-r Exp $");
+__KERNEL_RCSID(0, "$NetBSD: rtsock.c,v 1.199 2016/12/12 03:55:57 ozaki-r Exp $");
#ifdef _KERNEL_OPT
#include "opt_inet.h"
@@ -445,7 +445,7 @@ route_get_sdl_index(struct rt_addrinfo *
* due to changing to ifplo0.
*/
*sdl_index = satosdl(nrt->rt_gateway)->sdl_index;
- rtfree(nrt);
+ rt_unref(nrt);
return 0;
}
@@ -636,7 +636,8 @@ route_output_change(struct rtentry *rt,
}
if (ifa) {
struct ifaddr *oifa = rt->rt_ifa;
- if (oifa != ifa) {
+ if (oifa != ifa &&
+ !ifa_is_destroying(ifa) && !if_is_deactivated(new_ifp)) {
if (oifa && oifa->ifa_rtrequest)
oifa->ifa_rtrequest(RTM_DELETE, rt, info);
rt_replace_ifa(rt, ifa);
@@ -646,7 +647,8 @@ route_output_change(struct rtentry *rt,
ifa_release(ifa, &psref_ifa);
}
ifa_release(new_ifa, &psref_new_ifa);
- if (new_ifp && rt->rt_ifp != new_ifp)
+ if (new_ifp && rt->rt_ifp != new_ifp
+ && !if_is_deactivated(new_ifp))
rt->rt_ifp = new_ifp;
rt_setmetrics(rtm->rtm_inits, rtm, rt);
if (rt->rt_flags != info->rti_flags)
@@ -674,6 +676,7 @@ COMPATNAME(route_output)(struct mbuf *m,
sa_family_t family;
struct sockaddr_dl sdl;
int bound = curlwp_bind();
+ bool do_rt_free = false;
#define senderr(e) do { error = e; goto flush;} while (/*CONSTCOND*/ 0)
if (m == NULL || ((m->m_len < sizeof(int32_t)) &&
@@ -784,7 +787,7 @@ COMPATNAME(route_output)(struct mbuf *m,
error = rtrequest1(rtm->rtm_type, &info, &saved_nrt);
if (error == 0) {
rt_setmetrics(rtm->rtm_inits, rtm, saved_nrt);
- rtfree(saved_nrt);
+ rt_unref(saved_nrt);
}
break;
@@ -804,6 +807,7 @@ COMPATNAME(route_output)(struct mbuf *m,
break;
rt = saved_nrt;
+ do_rt_free = true;
info.rti_info[RTAX_DST] = rt_getkey(rt);
info.rti_info[RTAX_GATEWAY] = rt->rt_gateway;
info.rti_info[RTAX_NETMASK] = rt_mask(rt);
@@ -883,7 +887,11 @@ COMPATNAME(route_output)(struct mbuf *m,
break;
case RTM_CHANGE:
- error = route_output_change(rt, &info, rtm);
+ error = rt_update_prepare(rt);
+ if (error == 0) {
+ error = route_output_change(rt, &info, rtm);
+ rt_update_finish(rt);
+ }
if (error != 0)
goto flush;
/*FALLTHROUGH*/
@@ -914,8 +922,12 @@ flush:
*/
if (old_rtm != NULL)
Free(old_rtm);
- if (rt)
- rtfree(rt);
+ if (rt) {
+ if (do_rt_free)
+ rt_free(rt);
+ else
+ rt_unref(rt);
+ }
{
struct rawcb *rp = NULL;
/*
Index: src/sys/netinet/if_arp.c
diff -u src/sys/netinet/if_arp.c:1.232 src/sys/netinet/if_arp.c:1.233
--- src/sys/netinet/if_arp.c:1.232 Sat Nov 5 20:03:15 2016
+++ src/sys/netinet/if_arp.c Mon Dec 12 03:55:57 2016
@@ -1,4 +1,4 @@
-/* $NetBSD: if_arp.c,v 1.232 2016/11/05 20:03:15 roy Exp $ */
+/* $NetBSD: if_arp.c,v 1.233 2016/12/12 03:55:57 ozaki-r Exp $ */
/*-
* Copyright (c) 1998, 2000, 2008 The NetBSD Foundation, Inc.
@@ -68,7 +68,7 @@
*/
#include <sys/cdefs.h>
-__KERNEL_RCSID(0, "$NetBSD: if_arp.c,v 1.232 2016/11/05 20:03:15 roy Exp $");
+__KERNEL_RCSID(0, "$NetBSD: if_arp.c,v 1.233 2016/12/12 03:55:57 ozaki-r Exp $");
#ifdef _KERNEL_OPT
#include "opt_ddb.h"
@@ -894,7 +894,7 @@ notfound:
arprequest(ifp,
&satocsin(_rt->rt_ifa->ifa_addr)->sin_addr,
&satocsin(dst)->sin_addr, enaddr);
- rtfree(_rt);
+ rt_unref(_rt);
}
return error;
}
Index: src/sys/netinet/if_atm.c
diff -u src/sys/netinet/if_atm.c:1.38 src/sys/netinet/if_atm.c:1.39
--- src/sys/netinet/if_atm.c:1.38 Thu Apr 28 00:16:56 2016
+++ src/sys/netinet/if_atm.c Mon Dec 12 03:55:57 2016
@@ -1,4 +1,4 @@
-/* $NetBSD: if_atm.c,v 1.38 2016/04/28 00:16:56 ozaki-r Exp $ */
+/* $NetBSD: if_atm.c,v 1.39 2016/12/12 03:55:57 ozaki-r Exp $ */
/*
* Copyright (c) 1996 Charles D. Cranor and Washington University.
@@ -30,7 +30,7 @@
*/
#include <sys/cdefs.h>
-__KERNEL_RCSID(0, "$NetBSD: if_atm.c,v 1.38 2016/04/28 00:16:56 ozaki-r Exp $");
+__KERNEL_RCSID(0, "$NetBSD: if_atm.c,v 1.39 2016/12/12 03:55:57 ozaki-r Exp $");
#ifdef _KERNEL_OPT
#include "opt_inet.h"
@@ -219,7 +219,7 @@ atmresolve(const struct rtentry *rt0, st
if ((rt->rt_flags & RTF_GATEWAY) != 0 ||
/* XXX: are we using LLINFO? */
rt->rt_gateway->sa_family != AF_LINK) {
- rtfree(rt);
+ rt_unref(rt);
goto bad;
}
}
@@ -241,12 +241,12 @@ atmresolve(const struct rtentry *rt0, st
if (sdl->sdl_family == AF_LINK && sdl->sdl_alen == sizeof(*desten)) {
memcpy(desten, CLLADDR(sdl), sdl->sdl_alen);
if (rt != NULL)
- rtfree(rt);
+ rt_unref(rt);
return (1); /* ok, go for it! */
}
if (rt != NULL)
- rtfree(rt);
+ rt_unref(rt);
/*
* we got an entry, but it doesn't have valid link address
Index: src/sys/netinet/in.c
diff -u src/sys/netinet/in.c:1.190 src/sys/netinet/in.c:1.191
--- src/sys/netinet/in.c:1.190 Thu Dec 8 05:16:33 2016
+++ src/sys/netinet/in.c Mon Dec 12 03:55:57 2016
@@ -1,4 +1,4 @@
-/* $NetBSD: in.c,v 1.190 2016/12/08 05:16:33 ozaki-r Exp $ */
+/* $NetBSD: in.c,v 1.191 2016/12/12 03:55:57 ozaki-r Exp $ */
/*
* Copyright (C) 1995, 1996, 1997, and 1998 WIDE Project.
@@ -91,7 +91,7 @@
*/
#include <sys/cdefs.h>
-__KERNEL_RCSID(0, "$NetBSD: in.c,v 1.190 2016/12/08 05:16:33 ozaki-r Exp $");
+__KERNEL_RCSID(0, "$NetBSD: in.c,v 1.191 2016/12/12 03:55:57 ozaki-r Exp $");
#include "arp.h"
@@ -823,6 +823,7 @@ in_purgeaddr(struct ifaddr *ifa)
KASSERT(!ifa_held(ifa));
+ ifa->ifa_flags |= IFA_DESTROYING;
in_scrubaddr(ia);
mutex_enter(&in_ifaddr_lock);
@@ -2008,7 +2009,7 @@ in_lltable_rtcheck(struct ifnet *ifp, u_
error = 0;
error:
- rtfree(rt);
+ rt_unref(rt);
return error;
}
Index: src/sys/netinet/in_gif.c
diff -u src/sys/netinet/in_gif.c:1.84 src/sys/netinet/in_gif.c:1.85
--- src/sys/netinet/in_gif.c:1.84 Thu Dec 8 05:16:33 2016
+++ src/sys/netinet/in_gif.c Mon Dec 12 03:55:57 2016
@@ -1,4 +1,4 @@
-/* $NetBSD: in_gif.c,v 1.84 2016/12/08 05:16:33 ozaki-r Exp $ */
+/* $NetBSD: in_gif.c,v 1.85 2016/12/12 03:55:57 ozaki-r Exp $ */
/* $KAME: in_gif.c,v 1.66 2001/07/29 04:46:09 itojun Exp $ */
/*
@@ -31,7 +31,7 @@
*/
#include <sys/cdefs.h>
-__KERNEL_RCSID(0, "$NetBSD: in_gif.c,v 1.84 2016/12/08 05:16:33 ozaki-r Exp $");
+__KERNEL_RCSID(0, "$NetBSD: in_gif.c,v 1.85 2016/12/12 03:55:57 ozaki-r Exp $");
#ifdef _KERNEL_OPT
#include "opt_inet.h"
@@ -331,10 +331,10 @@ gif_validate4(const struct ip *ip, struc
(u_int32_t)ntohl(u.sin.sin_addr.s_addr));
#endif
if (rt != NULL)
- rtfree(rt);
+ rt_unref(rt);
return 0;
}
- rtfree(rt);
+ rt_unref(rt);
}
return 32 * 2;
Index: src/sys/netinet/in_pcb.c
diff -u src/sys/netinet/in_pcb.c:1.171 src/sys/netinet/in_pcb.c:1.172
--- src/sys/netinet/in_pcb.c:1.171 Thu Dec 8 05:16:33 2016
+++ src/sys/netinet/in_pcb.c Mon Dec 12 03:55:57 2016
@@ -1,4 +1,4 @@
-/* $NetBSD: in_pcb.c,v 1.171 2016/12/08 05:16:33 ozaki-r Exp $ */
+/* $NetBSD: in_pcb.c,v 1.172 2016/12/12 03:55:57 ozaki-r Exp $ */
/*
* Copyright (C) 1995, 1996, 1997, and 1998 WIDE Project.
@@ -93,7 +93,7 @@
*/
#include <sys/cdefs.h>
-__KERNEL_RCSID(0, "$NetBSD: in_pcb.c,v 1.171 2016/12/08 05:16:33 ozaki-r Exp $");
+__KERNEL_RCSID(0, "$NetBSD: in_pcb.c,v 1.172 2016/12/12 03:55:57 ozaki-r Exp $");
#ifdef _KERNEL_OPT
#include "opt_inet.h"
@@ -816,7 +816,7 @@ in_losing(struct inpcb *inp)
rt->rt_gateway, rt_mask(rt), rt->rt_flags, &nrt);
rtcache_unref(rt, &inp->inp_route);
if (error == 0)
- rtfree(nrt);
+ rt_free(nrt);
} else
rtcache_unref(rt, &inp->inp_route);
/*
Index: src/sys/netinet/ip_carp.c
diff -u src/sys/netinet/ip_carp.c:1.79 src/sys/netinet/ip_carp.c:1.80
--- src/sys/netinet/ip_carp.c:1.79 Tue Oct 11 13:39:34 2016
+++ src/sys/netinet/ip_carp.c Mon Dec 12 03:55:57 2016
@@ -1,4 +1,4 @@
-/* $NetBSD: ip_carp.c,v 1.79 2016/10/11 13:39:34 roy Exp $ */
+/* $NetBSD: ip_carp.c,v 1.80 2016/12/12 03:55:57 ozaki-r Exp $ */
/* $OpenBSD: ip_carp.c,v 1.113 2005/11/04 08:11:54 mcbride Exp $ */
/*
@@ -33,7 +33,7 @@
#endif
#include <sys/cdefs.h>
-__KERNEL_RCSID(0, "$NetBSD: ip_carp.c,v 1.79 2016/10/11 13:39:34 roy Exp $");
+__KERNEL_RCSID(0, "$NetBSD: ip_carp.c,v 1.80 2016/12/12 03:55:57 ozaki-r Exp $");
/*
* TODO:
@@ -397,7 +397,7 @@ carp_setroute(struct carp_softc *sc, int
hr_otherif = (rt && rt->rt_ifp != &sc->sc_if &&
(rt->rt_flags & RTF_CONNECTED));
if (rt != NULL) {
- rtfree(rt);
+ rt_unref(rt);
rt = NULL;
}
@@ -441,7 +441,7 @@ carp_setroute(struct carp_softc *sc, int
break;
}
if (rt != NULL) {
- rtfree(rt);
+ rt_unref(rt);
rt = NULL;
}
break;
Index: src/sys/netinet/ip_icmp.c
diff -u src/sys/netinet/ip_icmp.c:1.153 src/sys/netinet/ip_icmp.c:1.154
--- src/sys/netinet/ip_icmp.c:1.153 Tue Oct 25 02:45:09 2016
+++ src/sys/netinet/ip_icmp.c Mon Dec 12 03:55:57 2016
@@ -1,4 +1,4 @@
-/* $NetBSD: ip_icmp.c,v 1.153 2016/10/25 02:45:09 ozaki-r Exp $ */
+/* $NetBSD: ip_icmp.c,v 1.154 2016/12/12 03:55:57 ozaki-r Exp $ */
/*
* Copyright (C) 1995, 1996, 1997, and 1998 WIDE Project.
@@ -94,7 +94,7 @@
*/
#include <sys/cdefs.h>
-__KERNEL_RCSID(0, "$NetBSD: ip_icmp.c,v 1.153 2016/10/25 02:45:09 ozaki-r Exp $");
+__KERNEL_RCSID(0, "$NetBSD: ip_icmp.c,v 1.154 2016/12/12 03:55:57 ozaki-r Exp $");
#ifdef _KERNEL_OPT
#include "opt_ipsec.h"
@@ -652,7 +652,7 @@ reflect:
}
}
if (rt != NULL)
- rtfree(rt);
+ rt_unref(rt);
pfctlinput(PRC_REDIRECT_HOST, sintosa(&icmpsrc));
#if defined(IPSEC)
@@ -1148,16 +1148,19 @@ icmp_mtudisc(struct icmp *icp, struct in
error = rtrequest(RTM_ADD, dst, rt->rt_gateway, NULL,
RTF_GATEWAY | RTF_HOST | RTF_DYNAMIC, &nrt);
if (error) {
- rtfree(rt);
+ rt_unref(rt);
return;
}
nrt->rt_rmx = rt->rt_rmx;
- rtfree(rt);
+ rt_unref(rt);
rt = nrt;
}
+
+ if (ip_mtudisc_timeout_q == NULL)
+ ip_mtudisc_timeout_q = rt_timer_queue_create(ip_mtudisc_timeout);
error = rt_timer_add(rt, icmp_mtudisc_timeout, ip_mtudisc_timeout_q);
if (error) {
- rtfree(rt);
+ rt_unref(rt);
return;
}
@@ -1205,8 +1208,8 @@ icmp_mtudisc(struct icmp *icp, struct in
}
}
- if (rt)
- rtfree(rt);
+ if (rt != NULL)
+ rt_unref(rt);
/*
* Notify protocols that the MTU for this destination
Index: src/sys/netinet/ip_input.c
diff -u src/sys/netinet/ip_input.c:1.346 src/sys/netinet/ip_input.c:1.347
--- src/sys/netinet/ip_input.c:1.346 Thu Dec 8 06:25:43 2016
+++ src/sys/netinet/ip_input.c Mon Dec 12 03:55:57 2016
@@ -1,4 +1,4 @@
-/* $NetBSD: ip_input.c,v 1.346 2016/12/08 06:25:43 ozaki-r Exp $ */
+/* $NetBSD: ip_input.c,v 1.347 2016/12/12 03:55:57 ozaki-r Exp $ */
/*
* Copyright (C) 1995, 1996, 1997, and 1998 WIDE Project.
@@ -91,7 +91,7 @@
*/
#include <sys/cdefs.h>
-__KERNEL_RCSID(0, "$NetBSD: ip_input.c,v 1.346 2016/12/08 06:25:43 ozaki-r Exp $");
+__KERNEL_RCSID(0, "$NetBSD: ip_input.c,v 1.347 2016/12/12 03:55:57 ozaki-r Exp $");
#ifdef _KERNEL_OPT
#include "opt_inet.h"
@@ -335,7 +335,6 @@ ip_init(void)
ip_ids = ip_id_init();
ip_id = time_uptime & 0xfffff;
- ip_mtudisc_timeout_q = rt_timer_queue_create(ip_mtudisc_timeout);
#ifdef GATEWAY
ipflow_init();
#endif
Index: src/sys/netinet/ip_output.c
diff -u src/sys/netinet/ip_output.c:1.264 src/sys/netinet/ip_output.c:1.265
--- src/sys/netinet/ip_output.c:1.264 Thu Dec 8 05:16:33 2016
+++ src/sys/netinet/ip_output.c Mon Dec 12 03:55:57 2016
@@ -1,4 +1,4 @@
-/* $NetBSD: ip_output.c,v 1.264 2016/12/08 05:16:33 ozaki-r Exp $ */
+/* $NetBSD: ip_output.c,v 1.265 2016/12/12 03:55:57 ozaki-r Exp $ */
/*
* Copyright (C) 1995, 1996, 1997, and 1998 WIDE Project.
@@ -91,7 +91,7 @@
*/
#include <sys/cdefs.h>
-__KERNEL_RCSID(0, "$NetBSD: ip_output.c,v 1.264 2016/12/08 05:16:33 ozaki-r Exp $");
+__KERNEL_RCSID(0, "$NetBSD: ip_output.c,v 1.265 2016/12/12 03:55:57 ozaki-r Exp $");
#ifdef _KERNEL_OPT
#include "opt_inet.h"
@@ -361,10 +361,12 @@ ip_output(struct mbuf *m0, struct mbuf *
error = EHOSTUNREACH;
goto bad;
}
- /*
- * XXX NOMPSAFE: depends on accessing rt->rt_ifa isn't racy.
- * Revisit when working on rtentry MP-ification.
- */
+ if (ifa_is_destroying(rt->rt_ifa)) {
+ rtcache_unref(rt, ro);
+ IP_STATINC(IP_STAT_NOROUTE);
+ error = EHOSTUNREACH;
+ goto bad;
+ }
ifa_acquire(rt->rt_ifa, &psref_ia);
ia = ifatoia(rt->rt_ifa);
ifp = rt->rt_ifp;
Index: src/sys/netinet6/icmp6.c
diff -u src/sys/netinet6/icmp6.c:1.202 src/sys/netinet6/icmp6.c:1.203
--- src/sys/netinet6/icmp6.c:1.202 Sun Dec 11 07:35:42 2016
+++ src/sys/netinet6/icmp6.c Mon Dec 12 03:55:57 2016
@@ -1,4 +1,4 @@
-/* $NetBSD: icmp6.c,v 1.202 2016/12/11 07:35:42 ozaki-r Exp $ */
+/* $NetBSD: icmp6.c,v 1.203 2016/12/12 03:55:57 ozaki-r Exp $ */
/* $KAME: icmp6.c,v 1.217 2001/06/20 15:03:29 jinmei Exp $ */
/*
@@ -62,7 +62,7 @@
*/
#include <sys/cdefs.h>
-__KERNEL_RCSID(0, "$NetBSD: icmp6.c,v 1.202 2016/12/11 07:35:42 ozaki-r Exp $");
+__KERNEL_RCSID(0, "$NetBSD: icmp6.c,v 1.203 2016/12/12 03:55:57 ozaki-r Exp $");
#ifdef _KERNEL_OPT
#include "opt_inet.h"
@@ -1164,7 +1164,7 @@ icmp6_mtudisc_update(struct ip6ctlparam
}
}
if (rt) {
- rtfree(rt);
+ rt_unref(rt);
}
/*
@@ -2259,7 +2259,7 @@ icmp6_redirect_input(struct mbuf *m, int
"ICMP6 redirect rejected; no route "
"with inet6 gateway found for redirect dst: %s\n",
icmp6_redirect_diag(&src6, &reddst6, &redtgt6));
- rtfree(rt);
+ rt_unref(rt);
goto bad;
}
@@ -2270,7 +2270,7 @@ icmp6_redirect_input(struct mbuf *m, int
"not equal to gw-for-src=%s (must be same): %s\n",
ip6_sprintf(gw6),
icmp6_redirect_diag(&src6, &reddst6, &redtgt6));
- rtfree(rt);
+ rt_unref(rt);
goto bad;
}
} else {
@@ -2279,7 +2279,7 @@ icmp6_redirect_input(struct mbuf *m, int
icmp6_redirect_diag(&src6, &reddst6, &redtgt6));
goto bad;
}
- rtfree(rt);
+ rt_unref(rt);
rt = NULL;
}
if (IN6_IS_ADDR_MULTICAST(&reddst6)) {
@@ -2373,7 +2373,7 @@ icmp6_redirect_input(struct mbuf *m, int
if (newrt) {
(void)rt_timer_add(newrt, icmp6_redirect_timeout,
icmp6_redirect_timeout_q);
- rtfree(newrt);
+ rt_unref(newrt);
}
}
/* finally update cached route in each socket via pfctlinput */
@@ -2760,17 +2760,17 @@ icmp6_mtudisc_clone(struct sockaddr *dst
error = rtrequest(RTM_ADD, dst, rt->rt_gateway, NULL,
RTF_GATEWAY | RTF_HOST | RTF_DYNAMIC, &nrt);
if (error) {
- rtfree(rt);
+ rt_unref(rt);
return NULL;
}
nrt->rt_rmx = rt->rt_rmx;
- rtfree(rt);
+ rt_unref(rt);
rt = nrt;
}
error = rt_timer_add(rt, icmp6_mtudisc_timeout,
icmp6_mtudisc_timeout_q);
if (error) {
- rtfree(rt);
+ rt_unref(rt);
return NULL;
}
Index: src/sys/netinet6/in6.c
diff -u src/sys/netinet6/in6.c:1.223 src/sys/netinet6/in6.c:1.224
--- src/sys/netinet6/in6.c:1.223 Sun Dec 11 07:38:50 2016
+++ src/sys/netinet6/in6.c Mon Dec 12 03:55:57 2016
@@ -1,4 +1,4 @@
-/* $NetBSD: in6.c,v 1.223 2016/12/11 07:38:50 ozaki-r Exp $ */
+/* $NetBSD: in6.c,v 1.224 2016/12/12 03:55:57 ozaki-r Exp $ */
/* $KAME: in6.c,v 1.198 2001/07/18 09:12:38 itojun Exp $ */
/*
@@ -62,7 +62,7 @@
*/
#include <sys/cdefs.h>
-__KERNEL_RCSID(0, "$NetBSD: in6.c,v 1.223 2016/12/11 07:38:50 ozaki-r Exp $");
+__KERNEL_RCSID(0, "$NetBSD: in6.c,v 1.224 2016/12/12 03:55:57 ozaki-r Exp $");
#ifdef _KERNEL_OPT
#include "opt_inet.h"
@@ -1139,7 +1139,7 @@ in6_update_ifa1(struct ifnet *ifp, struc
if (memcmp(&mltaddr.sin6_addr,
&satocsin6(rt_getkey(rt))->sin6_addr,
MLTMASK_LEN)) {
- rtfree(rt);
+ rt_unref(rt);
rt = NULL;
} else if (rt->rt_ifp != ifp) {
IN6_DPRINTF("%s: rt_ifp %p -> %p (%s) "
@@ -1167,7 +1167,7 @@ in6_update_ifa1(struct ifnet *ifp, struc
if (error)
goto cleanup;
} else {
- rtfree(rt);
+ rt_unref(rt);
}
imm = in6_joingroup(ifp, &mltaddr.sin6_addr, &error, 0);
if (!imm) {
@@ -1220,7 +1220,7 @@ in6_update_ifa1(struct ifnet *ifp, struc
if (memcmp(&mltaddr.sin6_addr,
&satocsin6(rt_getkey(rt))->sin6_addr,
32 / NBBY)) {
- rtfree(rt);
+ rt_unref(rt);
rt = NULL;
} else if (rt->rt_ifp != ifp) {
IN6_DPRINTF("%s: rt_ifp %p -> %p (%s) "
@@ -1248,7 +1248,7 @@ in6_update_ifa1(struct ifnet *ifp, struc
goto cleanup;
#undef MLTMASK_LEN
} else {
- rtfree(rt);
+ rt_unref(rt);
}
imm = in6_joingroup(ifp, &mltaddr.sin6_addr, &error, 0);
if (!imm) {
@@ -1333,6 +1333,10 @@ in6_purgeaddr(struct ifaddr *ifa)
struct in6_ifaddr *ia = (struct in6_ifaddr *) ifa;
struct in6_multi_mship *imm;
+ KASSERT(!ifa_held(ifa));
+
+ ifa->ifa_flags |= IFA_DESTROYING;
+
/* stop DAD processing */
nd6_dad_stop(ifa);
@@ -2411,17 +2415,17 @@ in6_lltable_rtcheck(struct ifnet *ifp,
if (ifa != NULL) {
pserialize_read_exit(s);
if (rt != NULL)
- rtfree(rt);
+ rt_unref(rt);
return 0;
}
pserialize_read_exit(s);
log(LOG_INFO, "IPv6 address: \"%s\" is not on the network\n",
ip6_sprintf(&((const struct sockaddr_in6 *)l3addr)->sin6_addr));
if (rt != NULL)
- rtfree(rt);
+ rt_unref(rt);
return EINVAL;
}
- rtfree(rt);
+ rt_unref(rt);
return 0;
}
Index: src/sys/netinet6/in6_gif.c
diff -u src/sys/netinet6/in6_gif.c:1.80 src/sys/netinet6/in6_gif.c:1.81
--- src/sys/netinet6/in6_gif.c:1.80 Thu Dec 8 05:16:34 2016
+++ src/sys/netinet6/in6_gif.c Mon Dec 12 03:55:57 2016
@@ -1,4 +1,4 @@
-/* $NetBSD: in6_gif.c,v 1.80 2016/12/08 05:16:34 ozaki-r Exp $ */
+/* $NetBSD: in6_gif.c,v 1.81 2016/12/12 03:55:57 ozaki-r Exp $ */
/* $KAME: in6_gif.c,v 1.62 2001/07/29 04:27:25 itojun Exp $ */
/*
@@ -31,7 +31,7 @@
*/
#include <sys/cdefs.h>
-__KERNEL_RCSID(0, "$NetBSD: in6_gif.c,v 1.80 2016/12/08 05:16:34 ozaki-r Exp $");
+__KERNEL_RCSID(0, "$NetBSD: in6_gif.c,v 1.81 2016/12/12 03:55:57 ozaki-r Exp $");
#ifdef _KERNEL_OPT
#include "opt_inet.h"
@@ -333,10 +333,10 @@ gif_validate6(const struct ip6_hdr *ip6,
ip6_sprintf(&u.sin6.sin6_addr));
#endif
if (rt != NULL)
- rtfree(rt);
+ rt_unref(rt);
return 0;
}
- rtfree(rt);
+ rt_unref(rt);
}
return 128 * 2;
Index: src/sys/netinet6/in6_pcb.c
diff -u src/sys/netinet6/in6_pcb.c:1.153 src/sys/netinet6/in6_pcb.c:1.154
--- src/sys/netinet6/in6_pcb.c:1.153 Thu Dec 8 05:16:34 2016
+++ src/sys/netinet6/in6_pcb.c Mon Dec 12 03:55:57 2016
@@ -1,4 +1,4 @@
-/* $NetBSD: in6_pcb.c,v 1.153 2016/12/08 05:16:34 ozaki-r Exp $ */
+/* $NetBSD: in6_pcb.c,v 1.154 2016/12/12 03:55:57 ozaki-r Exp $ */
/* $KAME: in6_pcb.c,v 1.84 2001/02/08 18:02:08 itojun Exp $ */
/*
@@ -62,7 +62,7 @@
*/
#include <sys/cdefs.h>
-__KERNEL_RCSID(0, "$NetBSD: in6_pcb.c,v 1.153 2016/12/08 05:16:34 ozaki-r Exp $");
+__KERNEL_RCSID(0, "$NetBSD: in6_pcb.c,v 1.154 2016/12/12 03:55:57 ozaki-r Exp $");
#ifdef _KERNEL_OPT
#include "opt_inet.h"
@@ -930,7 +930,7 @@ in6_losing(struct in6pcb *in6p)
rt->rt_gateway, rt_mask(rt), rt->rt_flags, &nrt);
rtcache_unref(rt, &in6p->in6p_route);
if (error == 0)
- rtfree(nrt);
+ rt_free(nrt);
} else
rtcache_unref(rt, &in6p->in6p_route);
/*
Index: src/sys/netinet6/nd6.c
diff -u src/sys/netinet6/nd6.c:1.215 src/sys/netinet6/nd6.c:1.216
--- src/sys/netinet6/nd6.c:1.215 Mon Dec 12 03:14:01 2016
+++ src/sys/netinet6/nd6.c Mon Dec 12 03:55:57 2016
@@ -1,4 +1,4 @@
-/* $NetBSD: nd6.c,v 1.215 2016/12/12 03:14:01 ozaki-r Exp $ */
+/* $NetBSD: nd6.c,v 1.216 2016/12/12 03:55:57 ozaki-r Exp $ */
/* $KAME: nd6.c,v 1.279 2002/06/08 11:16:51 itojun Exp $ */
/*
@@ -31,7 +31,7 @@
*/
#include <sys/cdefs.h>
-__KERNEL_RCSID(0, "$NetBSD: nd6.c,v 1.215 2016/12/12 03:14:01 ozaki-r Exp $");
+__KERNEL_RCSID(0, "$NetBSD: nd6.c,v 1.216 2016/12/12 03:55:57 ozaki-r Exp $");
#ifdef _KERNEL_OPT
#include "opt_net_mpsafe.h"
@@ -1003,10 +1003,10 @@ nd6_is_new_addr_neighbor(const struct so
*/
if (!IN6_ARE_ADDR_EQUAL(&pr->ndpr_prefix.sin6_addr,
&satocsin6(rt_getkey(rt))->sin6_addr)) {
- rtfree(rt);
+ rt_unref(rt);
continue;
}
- rtfree(rt);
+ rt_unref(rt);
}
if (IN6_ARE_MASKED_ADDR_EQUAL(&pr->ndpr_prefix.sin6_addr,
@@ -1131,10 +1131,10 @@ nd6_is_addr_neighbor(const struct sockad
rt->rt_ifp->if_carpdev == ifp->if_carpdev)
#endif
)) {
- rtfree(rt);
+ rt_unref(rt);
return 1;
}
- rtfree(rt);
+ rt_unref(rt);
return 0;
}
Index: src/sys/netinet6/nd6_nbr.c
diff -u src/sys/netinet6/nd6_nbr.c:1.131 src/sys/netinet6/nd6_nbr.c:1.132
--- src/sys/netinet6/nd6_nbr.c:1.131 Sun Dec 11 07:38:50 2016
+++ src/sys/netinet6/nd6_nbr.c Mon Dec 12 03:55:57 2016
@@ -1,4 +1,4 @@
-/* $NetBSD: nd6_nbr.c,v 1.131 2016/12/11 07:38:50 ozaki-r Exp $ */
+/* $NetBSD: nd6_nbr.c,v 1.132 2016/12/12 03:55:57 ozaki-r Exp $ */
/* $KAME: nd6_nbr.c,v 1.61 2001/02/10 16:06:14 jinmei Exp $ */
/*
@@ -31,7 +31,7 @@
*/
#include <sys/cdefs.h>
-__KERNEL_RCSID(0, "$NetBSD: nd6_nbr.c,v 1.131 2016/12/11 07:38:50 ozaki-r Exp $");
+__KERNEL_RCSID(0, "$NetBSD: nd6_nbr.c,v 1.132 2016/12/12 03:55:57 ozaki-r Exp $");
#ifdef _KERNEL_OPT
#include "opt_inet.h"
@@ -256,7 +256,7 @@ nd6_ns_input(struct mbuf *m, int off, in
}
}
if (rt)
- rtfree(rt);
+ rt_unref(rt);
}
if (ifa == NULL) {
/*
