RE: [openib-general] Re: [PATCH 1/2] iWARP Connection Manager.
> > > > The problem is that we can't synchronously cancel an > > outstanding connect request. Once we've asked the adapter to > > connect, we can't tell him to stop, we have to wait for it to > > fail. During the time period between when we ask to connect > > and the adapter says yeah-or-nay, the user hits ctrl-C. This > > is the case where disconnect and/or destroy gets called and > > we have to block it waiting for the outstanding connect > > request to complete. > > > > One alternative to this approach is to do the kfree of the > > cm_id in the deref logic. This was the original design and > > leaves the object around to handle the completion of the > > connect and still allows the app to clean up and go away > > without all this waitin' around. When the adapter finally > > finishes and releases it's reference, the object is kfree'd. > > > > Hope this helps. > > > Why couldn't you synchronously put the cm_id in a state of > "pending delete" and do the actual delete when the RNIC > provides a response to the request? This is Tom's "alternative" mentioned above. The provider already keeps an explicit reference on the cm_id while it might possibly deliver an event on that cm_id. So if you change deref to kfree the cm_id on its last deref (when the refcnt reaches 0), then you can avoid blocking during destroy... > There could even be > an optional method to see if the device is capable of > cancelling the request. I know it can't yank a SYN back > from the wire, but it could refrain from retransmitting. I would suggest we don't add this optional method until we see an RNIC that supports canceling a connect request or accept synchronously... Steve. - To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
RE: [openib-general] Re: [PATCH 1/2] iWARP Connection Manager.
>> >> There's a difference between trying to handle the user calling >> disconnect/destroy at the same time a call to accept/connect is >> active, versus the user calling disconnect/destroy after >> accept/connect have returned. In the latter case, I think you're >> fine. In the first case, this is allowing a user to call > destroy at the same time that they're calling accept/connect. >> Additionally, there's no guarantee that the F_CONNECT_WAIT flag has >> been set by accept/connect by the time disconnect/destroy tests it. > > The problem is that we can't synchronously cancel an > outstanding connect request. Once we've asked the adapter to > connect, we can't tell him to stop, we have to wait for it to > fail. During the time period between when we ask to connect > and the adapter says yeah-or-nay, the user hits ctrl-C. This > is the case where disconnect and/or destroy gets called and > we have to block it waiting for the outstanding connect > request to complete. > > One alternative to this approach is to do the kfree of the > cm_id in the deref logic. This was the original design and > leaves the object around to handle the completion of the > connect and still allows the app to clean up and go away > without all this waitin' around. When the adapter finally > finishes and releases it's reference, the object is kfree'd. > > Hope this helps. > Why couldn't you synchronously put the cm_id in a state of "pending delete" and do the actual delete when the RNIC provides a response to the request? There could even be an optional method to see if the device is capable of cancelling the request. I know it can't yank a SYN back from the wire, but it could refrain from retransmitting. - To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [openib-general] Re: [PATCH 1/2] iWARP Connection Manager.
On Thu, 2006-06-01 at 14:09 -0700, Sean Hefty wrote:
> Steve Wise wrote:
> >>>+int iw_cm_disconnect(struct iw_cm_id *cm_id, int abrupt)
> >>>+{
> >>>+ struct iwcm_id_private *cm_id_priv;
> >>>+ unsigned long flags;
> >>>+ int ret = 0;
> >>>+
> >>>+ cm_id_priv = container_of(cm_id, struct iwcm_id_private, id);
> >>>+ /* Wait if we're currently in a connect or accept downcall */
> >>>+ wait_event(cm_id_priv->connect_wait,
> >>>+ !test_bit(IWCM_F_CONNECT_WAIT, &cm_id_priv->flags));
> >>
> >>Am I understanding this check correctly? You're checking to see if the
> >>user has
> >>called iw_cm_disconnect() at the same time that they called iw_cm_connect()
> >>or
> >>iw_cm_accept(). Are connect / accept blocking, or are you just waiting for
> >>an
> >>event?
> >
> >
> > The CM must wait for the low level provider to finish a connect() or
> > accept() operation before telling the low level provider to disconnect
> > via modifying the iwarp QP. Regardless of whether they block, this
> > disconnect can happen concurrently with the connect/accept so we need to
> > hold the disconnect until the connect/accept completes.
> >
> >
> >>>+EXPORT_SYMBOL(iw_cm_disconnect);
> >>>+static void destroy_cm_id(struct iw_cm_id *cm_id)
> >>>+{
> >>>+ struct iwcm_id_private *cm_id_priv;
> >>>+ unsigned long flags;
> >>>+ int ret;
> >>>+
> >>>+ cm_id_priv = container_of(cm_id, struct iwcm_id_private, id);
> >>>+ /* Wait if we're currently in a connect or accept downcall. A
> >>>+ * listening endpoint should never block here. */
> >>>+ wait_event(cm_id_priv->connect_wait,
> >>>+ !test_bit(IWCM_F_CONNECT_WAIT, &cm_id_priv->flags));
> >>
> >>Same question/comment as above.
> >>
> >
> >
> > Same answer.
>
> There's a difference between trying to handle the user calling
> disconnect/destroy at the same time a call to accept/connect is active,
> versus
> the user calling disconnect/destroy after accept/connect have returned. In
> the
> latter case, I think you're fine. In the first case, this is allowing a user
> to
> call destroy at the same time that they're calling accept/connect.
> Additionally, there's no guarantee that the F_CONNECT_WAIT flag has been set
> by
> accept/connect by the time disconnect/destroy tests it.
The problem is that we can't synchronously cancel an outstanding connect
request. Once we've asked the adapter to connect, we can't tell him to
stop, we have to wait for it to fail. During the time period between
when we ask to connect and the adapter says yeah-or-nay, the user hits
ctrl-C. This is the case where disconnect and/or destroy gets called and
we have to block it waiting for the outstanding connect request to
complete.
One alternative to this approach is to do the kfree of the cm_id in the
deref logic. This was the original design and leaves the object around
to handle the completion of the connect and still allows the app to
clean up and go away without all this waitin' around. When the adapter
finally finishes and releases it's reference, the object is kfree'd.
Hope this helps.
>
> - Sean
> ___
> openib-general mailing list
> openib-general@openib.org
> http://openib.org/mailman/listinfo/openib-general
>
> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
-
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 1/2] iWARP Connection Manager.
Steve Wise wrote:
+int iw_cm_disconnect(struct iw_cm_id *cm_id, int abrupt)
+{
+ struct iwcm_id_private *cm_id_priv;
+ unsigned long flags;
+ int ret = 0;
+
+ cm_id_priv = container_of(cm_id, struct iwcm_id_private, id);
+ /* Wait if we're currently in a connect or accept downcall */
+ wait_event(cm_id_priv->connect_wait,
+ !test_bit(IWCM_F_CONNECT_WAIT, &cm_id_priv->flags));
Am I understanding this check correctly? You're checking to see if the user has
called iw_cm_disconnect() at the same time that they called iw_cm_connect() or
iw_cm_accept(). Are connect / accept blocking, or are you just waiting for an
event?
The CM must wait for the low level provider to finish a connect() or
accept() operation before telling the low level provider to disconnect
via modifying the iwarp QP. Regardless of whether they block, this
disconnect can happen concurrently with the connect/accept so we need to
hold the disconnect until the connect/accept completes.
+EXPORT_SYMBOL(iw_cm_disconnect);
+static void destroy_cm_id(struct iw_cm_id *cm_id)
+{
+ struct iwcm_id_private *cm_id_priv;
+ unsigned long flags;
+ int ret;
+
+ cm_id_priv = container_of(cm_id, struct iwcm_id_private, id);
+ /* Wait if we're currently in a connect or accept downcall. A
+* listening endpoint should never block here. */
+ wait_event(cm_id_priv->connect_wait,
+ !test_bit(IWCM_F_CONNECT_WAIT, &cm_id_priv->flags));
Same question/comment as above.
Same answer.
There's a difference between trying to handle the user calling
disconnect/destroy at the same time a call to accept/connect is active, versus
the user calling disconnect/destroy after accept/connect have returned. In the
latter case, I think you're fine. In the first case, this is allowing a user to
call destroy at the same time that they're calling accept/connect.
Additionally, there's no guarantee that the F_CONNECT_WAIT flag has been set by
accept/connect by the time disconnect/destroy tests it.
- Sean
-
To unsubscribe from this list: send the line "unsubscribe netdev" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [openib-general] Re: [PATCH 1/2] iWARP Connection Manager.
On Wed, 2006-05-31 at 15:22 -0700, Sean Hefty wrote:
> Steve Wise wrote:
> > +/*
> > + * Release a reference on cm_id. If the last reference is being removed
> > + * and iw_destroy_cm_id is waiting, wake up the waiting thread.
> > + */
> > +static int iwcm_deref_id(struct iwcm_id_private *cm_id_priv)
> > +{
> > + int ret = 0;
> > +
> > + BUG_ON(atomic_read(&cm_id_priv->refcount)==0);
> > + if (atomic_dec_and_test(&cm_id_priv->refcount)) {
> > + BUG_ON(!list_empty(&cm_id_priv->work_list));
> > + if (waitqueue_active(&cm_id_priv->destroy_wait)) {
> > + BUG_ON(cm_id_priv->state != IW_CM_STATE_DESTROYING);
> > + BUG_ON(test_bit(IWCM_F_CALLBACK_DESTROY,
> > + &cm_id_priv->flags));
> > + ret = 1;
> > + wake_up(&cm_id_priv->destroy_wait);
>
> We recently changed the RDMA CM, IB CM, and a couple of other modules from
> using
> wait objects to completions. This avoids a race condition between
> decrementing
> the reference count, which allows destruction to proceed, and calling wake_up
> on
> a freed cm_id. My guess is that you may need to do the same.
>
> Can you also explain the use of the return value here? It's ignored below in
> rem_ref() and destroy_cm_id().
>
> > +static void add_ref(struct iw_cm_id *cm_id)
> > +{
> > + struct iwcm_id_private *cm_id_priv;
> > + cm_id_priv = container_of(cm_id, struct iwcm_id_private, id);
> > + atomic_inc(&cm_id_priv->refcount);
> > +}
> > +
> > +static void rem_ref(struct iw_cm_id *cm_id)
> > +{
> > + struct iwcm_id_private *cm_id_priv;
> > + cm_id_priv = container_of(cm_id, struct iwcm_id_private, id);
> > + iwcm_deref_id(cm_id_priv);
> > +}
> > +
>
> > +/*
> > + * CM_ID <-- CLOSING
> > + *
> > + * Block if a passive or active connection is currenlty being processed.
> > Then
> > + * process the event as follows:
> > + * - If we are ESTABLISHED, move to CLOSING and modify the QP state
> > + * based on the abrupt flag
> > + * - If the connection is already in the CLOSING or IDLE state, the peer is
> > + * disconnecting concurrently with us and we've already seen the
> > + * DISCONNECT event -- ignore the request and return 0
> > + * - Disconnect on a listening endpoint returns -EINVAL
> > + */
> > +int iw_cm_disconnect(struct iw_cm_id *cm_id, int abrupt)
> > +{
> > + struct iwcm_id_private *cm_id_priv;
> > + unsigned long flags;
> > + int ret = 0;
> > +
> > + cm_id_priv = container_of(cm_id, struct iwcm_id_private, id);
> > + /* Wait if we're currently in a connect or accept downcall */
> > + wait_event(cm_id_priv->connect_wait,
> > + !test_bit(IWCM_F_CONNECT_WAIT, &cm_id_priv->flags));
>
> Am I understanding this check correctly? You're checking to see if the user
> has
> called iw_cm_disconnect() at the same time that they called iw_cm_connect()
> or
> iw_cm_accept(). Are connect / accept blocking, or are you just waiting for
> an
> event?
Yes. The application (or the case I saw was user-mode exit logic after
ctrl-C) cleaning up at random times relative to connection
establishment.
>
> > +
> > + spin_lock_irqsave(&cm_id_priv->lock, flags);
> > + switch (cm_id_priv->state) {
> > + case IW_CM_STATE_ESTABLISHED:
> > + cm_id_priv->state = IW_CM_STATE_CLOSING;
> > + spin_unlock_irqrestore(&cm_id_priv->lock, flags);
> > + if (cm_id_priv->qp) { /* QP could be for user-mode
> > client */
> > + if (abrupt)
> > + ret = iwcm_modify_qp_err(cm_id_priv->qp);
> > + else
> > + ret = iwcm_modify_qp_sqd(cm_id_priv->qp);
> > + /*
> > +* If both sides are disconnecting the QP could
> > +* already be in ERR or SQD states
> > +*/
> > + ret = 0;
> > + }
> > + else
> > + ret = -EINVAL;
> > + break;
> > + case IW_CM_STATE_LISTEN:
> > + spin_unlock_irqrestore(&cm_id_priv->lock, flags);
> > + ret = -EINVAL;
> > + break;
> > + case IW_CM_STATE_CLOSING:
> > + /* remote peer closed first */
> > + case IW_CM_STATE_IDLE:
> > + /* accept or connect returned !0 */
> > + spin_unlock_irqrestore(&cm_id_priv->lock, flags);
> > + break;
> > + case IW_CM_STATE_CONN_RECV:
> > + /*
> > +* App called disconnect before/without calling accept after
> > +* connect_request event delivered.
> > +*/
> > + spin_unlock_irqrestore(&cm_id_priv->lock, flags);
> > + break;
> > + case IW_CM_STATE_CONN_SENT:
> > + /* Can only get here if wait above fails */
> > + default:
> > + BUG_ON(1);
> > + }
> > +
> > + return ret;
> > +}
> > +EXPORT_SYMBOL(iw_cm_disconnect);
>
Re: [PATCH 1/2] iWARP Connection Manager.
On Wed, 2006-05-31 at 15:22 -0700, Sean Hefty wrote:
> Steve Wise wrote:
> > +/*
> > + * Release a reference on cm_id. If the last reference is being removed
> > + * and iw_destroy_cm_id is waiting, wake up the waiting thread.
> > + */
> > +static int iwcm_deref_id(struct iwcm_id_private *cm_id_priv)
> > +{
> > + int ret = 0;
> > +
> > + BUG_ON(atomic_read(&cm_id_priv->refcount)==0);
> > + if (atomic_dec_and_test(&cm_id_priv->refcount)) {
> > + BUG_ON(!list_empty(&cm_id_priv->work_list));
> > + if (waitqueue_active(&cm_id_priv->destroy_wait)) {
> > + BUG_ON(cm_id_priv->state != IW_CM_STATE_DESTROYING);
> > + BUG_ON(test_bit(IWCM_F_CALLBACK_DESTROY,
> > + &cm_id_priv->flags));
> > + ret = 1;
> > + wake_up(&cm_id_priv->destroy_wait);
>
> We recently changed the RDMA CM, IB CM, and a couple of other modules from
> using
> wait objects to completions. This avoids a race condition between
> decrementing
> the reference count, which allows destruction to proceed, and calling wake_up
> on
> a freed cm_id. My guess is that you may need to do the same.
>
Good catch. Yes, the IW CM suffers from the same race condition. I'll
change this to use completions...
> Can you also explain the use of the return value here? It's ignored below in
> rem_ref() and destroy_cm_id().
>
The return value is supposed to indicate whether this call to deref
_may_ have resulted in waking up another thread and the cm_id being
freed. Its used in cm_work_handler(), in conjunction with setting the
IWCM_F_CALLBACK_DESTROY flag to know whether the cm_id needs to be freed
on the callback path.
> > +static void add_ref(struct iw_cm_id *cm_id)
> > +{
> > + struct iwcm_id_private *cm_id_priv;
> > + cm_id_priv = container_of(cm_id, struct iwcm_id_private, id);
> > + atomic_inc(&cm_id_priv->refcount);
> > +}
> > +
> > +static void rem_ref(struct iw_cm_id *cm_id)
> > +{
> > + struct iwcm_id_private *cm_id_priv;
> > + cm_id_priv = container_of(cm_id, struct iwcm_id_private, id);
> > + iwcm_deref_id(cm_id_priv);
> > +}
> > +
>
> > +/*
> > + * CM_ID <-- CLOSING
> > + *
> > + * Block if a passive or active connection is currenlty being processed.
> > Then
> > + * process the event as follows:
> > + * - If we are ESTABLISHED, move to CLOSING and modify the QP state
> > + * based on the abrupt flag
> > + * - If the connection is already in the CLOSING or IDLE state, the peer is
> > + * disconnecting concurrently with us and we've already seen the
> > + * DISCONNECT event -- ignore the request and return 0
> > + * - Disconnect on a listening endpoint returns -EINVAL
> > + */
> > +int iw_cm_disconnect(struct iw_cm_id *cm_id, int abrupt)
> > +{
> > + struct iwcm_id_private *cm_id_priv;
> > + unsigned long flags;
> > + int ret = 0;
> > +
> > + cm_id_priv = container_of(cm_id, struct iwcm_id_private, id);
> > + /* Wait if we're currently in a connect or accept downcall */
> > + wait_event(cm_id_priv->connect_wait,
> > + !test_bit(IWCM_F_CONNECT_WAIT, &cm_id_priv->flags));
>
> Am I understanding this check correctly? You're checking to see if the user
> has
> called iw_cm_disconnect() at the same time that they called iw_cm_connect()
> or
> iw_cm_accept(). Are connect / accept blocking, or are you just waiting for
> an
> event?
The CM must wait for the low level provider to finish a connect() or
accept() operation before telling the low level provider to disconnect
via modifying the iwarp QP. Regardless of whether they block, this
disconnect can happen concurrently with the connect/accept so we need to
hold the disconnect until the connect/accept completes.
>
> > +
> > + spin_lock_irqsave(&cm_id_priv->lock, flags);
> > + switch (cm_id_priv->state) {
> > + case IW_CM_STATE_ESTABLISHED:
> > + cm_id_priv->state = IW_CM_STATE_CLOSING;
> > + spin_unlock_irqrestore(&cm_id_priv->lock, flags);
> > + if (cm_id_priv->qp) { /* QP could be for user-mode
> > client */
> > + if (abrupt)
> > + ret = iwcm_modify_qp_err(cm_id_priv->qp);
> > + else
> > + ret = iwcm_modify_qp_sqd(cm_id_priv->qp);
> > + /*
> > +* If both sides are disconnecting the QP could
> > +* already be in ERR or SQD states
> > +*/
> > + ret = 0;
> > + }
> > + else
> > + ret = -EINVAL;
> > + break;
> > + case IW_CM_STATE_LISTEN:
> > + spin_unlock_irqrestore(&cm_id_priv->lock, flags);
> > + ret = -EINVAL;
> > + break;
> > + case IW_CM_STATE_CLOSING:
> > + /* remote peer closed first */
> > + case IW_CM_STATE_IDLE:
> > + /* accept or connect returned !0 */
> > + sp
Re: [PATCH 1/2] iWARP Connection Manager.
Steve Wise wrote:
+/*
+ * Release a reference on cm_id. If the last reference is being removed
+ * and iw_destroy_cm_id is waiting, wake up the waiting thread.
+ */
+static int iwcm_deref_id(struct iwcm_id_private *cm_id_priv)
+{
+ int ret = 0;
+
+ BUG_ON(atomic_read(&cm_id_priv->refcount)==0);
+ if (atomic_dec_and_test(&cm_id_priv->refcount)) {
+ BUG_ON(!list_empty(&cm_id_priv->work_list));
+ if (waitqueue_active(&cm_id_priv->destroy_wait)) {
+ BUG_ON(cm_id_priv->state != IW_CM_STATE_DESTROYING);
+ BUG_ON(test_bit(IWCM_F_CALLBACK_DESTROY,
+ &cm_id_priv->flags));
+ ret = 1;
+ wake_up(&cm_id_priv->destroy_wait);
We recently changed the RDMA CM, IB CM, and a couple of other modules from using
wait objects to completions. This avoids a race condition between decrementing
the reference count, which allows destruction to proceed, and calling wake_up on
a freed cm_id. My guess is that you may need to do the same.
Can you also explain the use of the return value here? It's ignored below in
rem_ref() and destroy_cm_id().
+static void add_ref(struct iw_cm_id *cm_id)
+{
+ struct iwcm_id_private *cm_id_priv;
+ cm_id_priv = container_of(cm_id, struct iwcm_id_private, id);
+ atomic_inc(&cm_id_priv->refcount);
+}
+
+static void rem_ref(struct iw_cm_id *cm_id)
+{
+ struct iwcm_id_private *cm_id_priv;
+ cm_id_priv = container_of(cm_id, struct iwcm_id_private, id);
+ iwcm_deref_id(cm_id_priv);
+}
+
+/*
+ * CM_ID <-- CLOSING
+ *
+ * Block if a passive or active connection is currenlty being processed. Then
+ * process the event as follows:
+ * - If we are ESTABLISHED, move to CLOSING and modify the QP state
+ * based on the abrupt flag
+ * - If the connection is already in the CLOSING or IDLE state, the peer is
+ * disconnecting concurrently with us and we've already seen the
+ * DISCONNECT event -- ignore the request and return 0
+ * - Disconnect on a listening endpoint returns -EINVAL
+ */
+int iw_cm_disconnect(struct iw_cm_id *cm_id, int abrupt)
+{
+ struct iwcm_id_private *cm_id_priv;
+ unsigned long flags;
+ int ret = 0;
+
+ cm_id_priv = container_of(cm_id, struct iwcm_id_private, id);
+ /* Wait if we're currently in a connect or accept downcall */
+ wait_event(cm_id_priv->connect_wait,
+ !test_bit(IWCM_F_CONNECT_WAIT, &cm_id_priv->flags));
Am I understanding this check correctly? You're checking to see if the user has
called iw_cm_disconnect() at the same time that they called iw_cm_connect() or
iw_cm_accept(). Are connect / accept blocking, or are you just waiting for an
event?
+
+ spin_lock_irqsave(&cm_id_priv->lock, flags);
+ switch (cm_id_priv->state) {
+ case IW_CM_STATE_ESTABLISHED:
+ cm_id_priv->state = IW_CM_STATE_CLOSING;
+ spin_unlock_irqrestore(&cm_id_priv->lock, flags);
+ if (cm_id_priv->qp) { /* QP could be for user-mode
client */
+ if (abrupt)
+ ret = iwcm_modify_qp_err(cm_id_priv->qp);
+ else
+ ret = iwcm_modify_qp_sqd(cm_id_priv->qp);
+ /*
+ * If both sides are disconnecting the QP could
+* already be in ERR or SQD states
+*/
+ ret = 0;
+ }
+ else
+ ret = -EINVAL;
+ break;
+ case IW_CM_STATE_LISTEN:
+ spin_unlock_irqrestore(&cm_id_priv->lock, flags);
+ ret = -EINVAL;
+ break;
+ case IW_CM_STATE_CLOSING:
+ /* remote peer closed first */
+ case IW_CM_STATE_IDLE:
+ /* accept or connect returned !0 */
+ spin_unlock_irqrestore(&cm_id_priv->lock, flags);
+ break;
+ case IW_CM_STATE_CONN_RECV:
+ /*
+ * App called disconnect before/without calling accept after
+* connect_request event delivered.
+*/
+ spin_unlock_irqrestore(&cm_id_priv->lock, flags);
+ break;
+ case IW_CM_STATE_CONN_SENT:
+ /* Can only get here if wait above fails */
+ default:
+ BUG_ON(1);
+ }
+
+ return ret;
+}
+EXPORT_SYMBOL(iw_cm_disconnect);
+static void destroy_cm_id(struct iw_cm_id *cm_id)
+{
+ struct iwcm_id_private *cm_id_priv;
+ unsigned long flags;
+ int ret;
+
+ cm_id_priv = container_of(cm_id, struct iwcm_id_private, id);
+ /* Wait if we're currently in a connect or accept downcall. A
+* listening endpoint should never block here. */
+ wait_event(cm_id_priv->connect_wait,
+ !test_bit(IWCM_F_CONNECT_WAIT, &cm_id_priv->flags));
Same question/comment as abo
[PATCH 1/2] iWARP Connection Manager.
This patch provides the new files implementing the iWARP Connection
Manager.
---
drivers/infiniband/core/iwcm.c | 887
include/rdma/iw_cm.h | 254 +++
include/rdma/iw_cm_private.h | 62 +++
3 files changed, 1203 insertions(+), 0 deletions(-)
diff --git a/drivers/infiniband/core/iwcm.c b/drivers/infiniband/core/iwcm.c
new file mode 100644
index 000..5657ee8
--- /dev/null
+++ b/drivers/infiniband/core/iwcm.c
@@ -0,0 +1,887 @@
+/*
+ * Copyright (c) 2004, 2005 Intel Corporation. All rights reserved.
+ * Copyright (c) 2004 Topspin Corporation. All rights reserved.
+ * Copyright (c) 2004, 2005 Voltaire Corporation. All rights reserved.
+ * Copyright (c) 2005 Sun Microsystems, Inc. All rights reserved.
+ * Copyright (c) 2005 Open Grid Computing, Inc. All rights reserved.
+ * Copyright (c) 2005 Network Appliance, Inc. All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses. You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ * Redistribution and use in source and binary forms, with or
+ * without modification, are permitted provided that the following
+ * conditions are met:
+ *
+ * - Redistributions of source code must retain the above
+ *copyright notice, this list of conditions and the following
+ *disclaimer.
+ *
+ * - Redistributions in binary form must reproduce the above
+ *copyright notice, this list of conditions and the following
+ *disclaimer in the documentation and/or other materials
+ *provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ *
+ */
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+MODULE_AUTHOR("Tom Tucker");
+MODULE_DESCRIPTION("iWARP CM");
+MODULE_LICENSE("Dual BSD/GPL");
+
+static struct workqueue_struct *iwcm_wq;
+struct iwcm_work {
+ struct work_struct work;
+ struct iwcm_id_private *cm_id;
+ struct list_head list;
+ struct iw_cm_event event;
+};
+
+/*
+ * Release a reference on cm_id. If the last reference is being removed
+ * and iw_destroy_cm_id is waiting, wake up the waiting thread.
+ */
+static int iwcm_deref_id(struct iwcm_id_private *cm_id_priv)
+{
+ int ret = 0;
+
+ BUG_ON(atomic_read(&cm_id_priv->refcount)==0);
+ if (atomic_dec_and_test(&cm_id_priv->refcount)) {
+ BUG_ON(!list_empty(&cm_id_priv->work_list));
+ if (waitqueue_active(&cm_id_priv->destroy_wait)) {
+ BUG_ON(cm_id_priv->state != IW_CM_STATE_DESTROYING);
+ BUG_ON(test_bit(IWCM_F_CALLBACK_DESTROY,
+ &cm_id_priv->flags));
+ ret = 1;
+ wake_up(&cm_id_priv->destroy_wait);
+ }
+ }
+
+ return ret;
+}
+
+static void add_ref(struct iw_cm_id *cm_id)
+{
+ struct iwcm_id_private *cm_id_priv;
+ cm_id_priv = container_of(cm_id, struct iwcm_id_private, id);
+ atomic_inc(&cm_id_priv->refcount);
+}
+
+static void rem_ref(struct iw_cm_id *cm_id)
+{
+ struct iwcm_id_private *cm_id_priv;
+ cm_id_priv = container_of(cm_id, struct iwcm_id_private, id);
+ iwcm_deref_id(cm_id_priv);
+}
+
+static void cm_event_handler(struct iw_cm_id *cm_id, struct iw_cm_event
*event);
+
+struct iw_cm_id *iw_create_cm_id(struct ib_device *device,
+iw_cm_handler cm_handler,
+void *context)
+{
+ struct iwcm_id_private *cm_id_priv;
+
+ cm_id_priv = kzalloc(sizeof *cm_id_priv, GFP_KERNEL);
+ if (!cm_id_priv)
+ return ERR_PTR(-ENOMEM);
+
+ cm_id_priv->state = IW_CM_STATE_IDLE;
+ cm_id_priv->id.device = device;
+ cm_id_priv->id.cm_handler = cm_handler;
+ cm_id_priv->id.context = context;
+ cm_id_priv->id.event_handler = cm_event_handler;
+ cm_id_priv->id.add_ref = add_ref;
+ cm_id_priv->id.rem_ref = rem_ref;
+ spin_lock_init(&cm_id_priv->lock);
+ atomic_set(&cm_id_priv->refcount, 1);
+ init_waitqueue_head(&cm_id_priv->connect_wait);
+ init_waitqueue_head(&cm_id_priv->destroy_wait);
+ INIT_LIST_HEAD(&cm_id_priv->work_list);
+
+ return &c
