Re: [PATCH 4/4] target: Add a user-passthrough backstore

2014-09-22 Thread Nicholas A. Bellinger
On Mon, 2014-09-22 at 14:00 -0700, Andy Grover wrote:
> On 09/22/2014 01:58 PM, Nicholas A. Bellinger wrote:
> > So I'd still like to start for an initial merge with the two different
> > modes mentioned earlier.  The pure-passthrough mode where everything is
> > handled by user-space, and an I/O passthrough mode where only
> > SCF_SCSI_DATA_CDB is passed along to userspace, but all other control
> > CDBs are handled by existing in-kernel emulation.
> >
> > Andy, can you re-spin a series with these two approaches in mind..?
> 
> Will do. This is actually close to what the initial RFC version I posted 
> did.
> 

Also, it would be nice to include the example user-space code that
proves the logic into the patch series, so it can be carried along with
kernel sources.

--nab

--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 4/4] target: Add a user-passthrough backstore

2014-09-22 Thread Andy Grover

On 09/22/2014 01:58 PM, Nicholas A. Bellinger wrote:

So I'd still like to start for an initial merge with the two different
modes mentioned earlier.  The pure-passthrough mode where everything is
handled by user-space, and an I/O passthrough mode where only
SCF_SCSI_DATA_CDB is passed along to userspace, but all other control
CDBs are handled by existing in-kernel emulation.

Andy, can you re-spin a series with these two approaches in mind..?


Will do. This is actually close to what the initial RFC version I posted 
did.


-- Andy


--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 4/4] target: Add a user-passthrough backstore

2014-09-22 Thread Nicholas A. Bellinger
On Fri, 2014-09-19 at 17:35 -0700, Andy Grover wrote:
> On 09/19/2014 04:51 PM, Alex Elsayed wrote:
> 
> >> Not sure I follow..  How does the proposed passthrough mode prevent
> >> someone from emulating OSDs, media changers, optical disks or anything
> >> else in userspace with TCMU..?
> >>
> >> The main thing that the above comments highlight is why attempting to
> >> combine the existing in-kernel emulation with a userspace backend
> >> providing it's own emulation can open up a number of problems with
> >> mismatched state between the two.
> >
> > It doesn't prevent it, but it _does_ put it in the exact same place as PSCSI
> > regarding the warnings on the wiki. It means that if someone wants to
> > implement (say) the optical disc or OSD CDBs, they then lose out on ALUA &co
> > unless they implement it themselves - which seems unnecessary and painful,
> > since those should really be disjoint. In particular, an OSD backed by RADOS
> > objects could be a very nice thing indeed, _and_ could really benefit from
> > ALUA.
> 
> Some possible solutions:
> 
> 1) The first time TCMU sees an opcode it passes it up and notes whether 
> it is handled or not. If it was handled then future cmds with that 
> opcode are passed up but not retried in the kernel. If it was not 
> handled then it and all future commands with that opcode are handled by 
> the kernel and not passed up.
> 

I'd rather avoid having to keep around a bunch of extra per-device
state, and having to send all opcodes out to user-space first is messy.

> 2) Same as #1 but TCMU operates on SCSI spec granularity - e.g. handling 
> any SSC opcode means userspace must handle all SSC commands.
> 

It's unclear how effective this will ultimately be, given that some
different standards include EVPD and/or mode pages specific to the
peripheral type.

> 3) Like #2 but define opcode groupings that must all be implemented 
> together, independent of the specifications.
> 

This is overly complex, and defining the groups of opcodes is
problematic to maintain.

> 4) Have passthrough mode set at creation, but with more than two modes, 
> either grouped by SCSI spec or our own groupings.
> 

Passthrough mode set at creation is the correct default, but grouping
things by SCSI spec and custom opcode groupings doesn't really make alot
of sense to me.

> 5) Never pass up certain opcodes, like the ALUA ones or whatever.
> 

This hurts flexibility, given that some folks might already have their
own ALUA implemented in a user-space driver, and would like to be able
to run in pure passthrough mode.

So I'd still like to start for an initial merge with the two different
modes mentioned earlier.  The pure-passthrough mode where everything is
handled by user-space, and an I/O passthrough mode where only
SCF_SCSI_DATA_CDB is passed along to userspace, but all other control
CDBs are handled by existing in-kernel emulation.

Andy, can you re-spin a series with these two approaches in mind..?

--nab

--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 4/4] target: Add a user-passthrough backstore

2014-09-22 Thread Nicholas A. Bellinger
On Fri, 2014-09-19 at 16:51 -0700, Alex Elsayed wrote:
> Nicholas A. Bellinger wrote:
> 
> > On Fri, 2014-09-19 at 14:43 -0700, Alex Elsayed wrote:
> >> Nicholas A. Bellinger wrote:
> >> 
> >> 
> >> > So the idea of allowing the in-kernel CDB emulation to run after
> >> > user-space has returned unsupported opcode is problematic for a couple
> >> > of different reasons.
> >> > 
> >> > First, if the correct feature bits in standard INQUIRY + EVPD INQUIRY,
> >> > etc are not populated by user-space to match what in-kernel CDB
> >> > emulation actually supports, this can result in potentially undefined
> >> > results initiator side.
> >> > 
> >> > Also for example, if user-space decided to emulate only a subset of PR
> >> > operations, and leaves the rest of it up to the in-kernel emulation,
> >> > there's not really a way to sync current state between the two, which
> >> > also can end up with undefined results.
> >> > 
> >> > So that said, I think a saner approach would be two define two modes of
> >> > operation for TCMU:
> >> > 
> >> >*) Passthrough Mode: All CDBs are passed to user-space, and no
> >> >   in-kernel emulation is done in the event of an unsupported
> >> >   opcode response.
> >> > 
> >> >*) I/O Mode: I/O related CDBs are passed into user-space, but
> >> >   all control CDBs continue to be processed by in-kernel emulation.
> >> >   This effectively limits operation to TYPE_DISK, but with this
> >> >   mode it's probably OK to assume this.
> >> > 
> >> > This seems like the best trade-off between flexibility when everything
> >> > should be handled by user-space, vs. functionality when only block
> >> > remapping of I/O is occurring in user-space code.
> >> 
> >> The problem there is that the first case has all the issues of pscsi and
> >> simply becomes a performance optimization over tgt+iscsi client+pscsi and
> >> the latter case excludes the main use cases I'm interested in - OSDs,
> >> media changers, optical discs (the biggest thing for me), and so forth.
> >> 
> >> One of the main things I want to do with this is hook up a plugin that
> >> uses libmirage to handle various optical disc image formats.
> >> 
> > 
> > Not sure I follow..  How does the proposed passthrough mode prevent
> > someone from emulating OSDs, media changers, optical disks or anything
> > else in userspace with TCMU..?
> > 
> > The main thing that the above comments highlight is why attempting to
> > combine the existing in-kernel emulation with a userspace backend
> > providing it's own emulation can open up a number of problems with
> > mismatched state between the two.
> 
> It doesn't prevent it, but it _does_ put it in the exact same place as PSCSI 
> regarding the warnings on the wiki.

Not exactly.  PSCSI has no mode where control commands are handled by
the in-kernel emulation.

> It means that if someone wants to 
> implement (say) the optical disc or OSD CDBs, they then lose out on ALUA &co 
> unless they implement it themselves - which seems unnecessary and painful, 
> since those should really be disjoint. In particular, an OSD backed by RADOS 
> objects could be a very nice thing indeed, _and_ could really benefit from 
> ALUA.
> 

For OSD strictly speaking in the T10 sense, the use-case your describing
would fall under the guise of I/O mode, where the OSD specific CDBs
would be propagated into the user-space backend driver.  This would
probably be OK, and the only thing that would need to change from the
perspective of existing in-kernel emulation is to expose TYPE_OSD
instead of TYPE_DISK.  Eg: AFAICT, there aren't any OSD specific EVPD or
mode pages.

As for optical discs, things are more complex because of mode pages
specific to MMC that effectively make it difficult to mark a clear
delineation between a hybrid I/O mode approach where some CDBs are
handed by in-kernel emulation, and some are handled in user-space.

So that said, an I/O mode that passes through OSD specific CDBs into
user-space should allow ALUA to still work as expected for TYPE_OSD,
where with optical drives it's less clear how to make a clean split, or
if anyone actually cares enough about ALUA emulation with TYPE_ROM.

--nab

--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 4/4] target: Add a user-passthrough backstore

2014-09-19 Thread Andy Grover

On 09/19/2014 04:51 PM, Alex Elsayed wrote:


Not sure I follow..  How does the proposed passthrough mode prevent
someone from emulating OSDs, media changers, optical disks or anything
else in userspace with TCMU..?

The main thing that the above comments highlight is why attempting to
combine the existing in-kernel emulation with a userspace backend
providing it's own emulation can open up a number of problems with
mismatched state between the two.


It doesn't prevent it, but it _does_ put it in the exact same place as PSCSI
regarding the warnings on the wiki. It means that if someone wants to
implement (say) the optical disc or OSD CDBs, they then lose out on ALUA &co
unless they implement it themselves - which seems unnecessary and painful,
since those should really be disjoint. In particular, an OSD backed by RADOS
objects could be a very nice thing indeed, _and_ could really benefit from
ALUA.


Some possible solutions:

1) The first time TCMU sees an opcode it passes it up and notes whether 
it is handled or not. If it was handled then future cmds with that 
opcode are passed up but not retried in the kernel. If it was not 
handled then it and all future commands with that opcode are handled by 
the kernel and not passed up.


2) Same as #1 but TCMU operates on SCSI spec granularity - e.g. handling 
any SSC opcode means userspace must handle all SSC commands.


3) Like #2 but define opcode groupings that must all be implemented 
together, independent of the specifications.


4) Have passthrough mode set at creation, but with more than two modes, 
either grouped by SCSI spec or our own groupings.


5) Never pass up certain opcodes, like the ALUA ones or whatever.


Have a good weekend -- Andy

--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 4/4] target: Add a user-passthrough backstore

2014-09-19 Thread Alex Elsayed
Nicholas A. Bellinger wrote:

> On Fri, 2014-09-19 at 14:43 -0700, Alex Elsayed wrote:
>> Nicholas A. Bellinger wrote:
>> 
>> 
>> > So the idea of allowing the in-kernel CDB emulation to run after
>> > user-space has returned unsupported opcode is problematic for a couple
>> > of different reasons.
>> > 
>> > First, if the correct feature bits in standard INQUIRY + EVPD INQUIRY,
>> > etc are not populated by user-space to match what in-kernel CDB
>> > emulation actually supports, this can result in potentially undefined
>> > results initiator side.
>> > 
>> > Also for example, if user-space decided to emulate only a subset of PR
>> > operations, and leaves the rest of it up to the in-kernel emulation,
>> > there's not really a way to sync current state between the two, which
>> > also can end up with undefined results.
>> > 
>> > So that said, I think a saner approach would be two define two modes of
>> > operation for TCMU:
>> > 
>> >*) Passthrough Mode: All CDBs are passed to user-space, and no
>> >   in-kernel emulation is done in the event of an unsupported
>> >   opcode response.
>> > 
>> >*) I/O Mode: I/O related CDBs are passed into user-space, but
>> >   all control CDBs continue to be processed by in-kernel emulation.
>> >   This effectively limits operation to TYPE_DISK, but with this
>> >   mode it's probably OK to assume this.
>> > 
>> > This seems like the best trade-off between flexibility when everything
>> > should be handled by user-space, vs. functionality when only block
>> > remapping of I/O is occurring in user-space code.
>> 
>> The problem there is that the first case has all the issues of pscsi and
>> simply becomes a performance optimization over tgt+iscsi client+pscsi and
>> the latter case excludes the main use cases I'm interested in - OSDs,
>> media changers, optical discs (the biggest thing for me), and so forth.
>> 
>> One of the main things I want to do with this is hook up a plugin that
>> uses libmirage to handle various optical disc image formats.
>> 
> 
> Not sure I follow..  How does the proposed passthrough mode prevent
> someone from emulating OSDs, media changers, optical disks or anything
> else in userspace with TCMU..?
> 
> The main thing that the above comments highlight is why attempting to
> combine the existing in-kernel emulation with a userspace backend
> providing it's own emulation can open up a number of problems with
> mismatched state between the two.

It doesn't prevent it, but it _does_ put it in the exact same place as PSCSI 
regarding the warnings on the wiki. It means that if someone wants to 
implement (say) the optical disc or OSD CDBs, they then lose out on ALUA &co 
unless they implement it themselves - which seems unnecessary and painful, 
since those should really be disjoint. In particular, an OSD backed by RADOS 
objects could be a very nice thing indeed, _and_ could really benefit from 
ALUA.

--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 4/4] target: Add a user-passthrough backstore

2014-09-19 Thread Nicholas A. Bellinger
On Fri, 2014-09-19 at 14:43 -0700, Alex Elsayed wrote:
> Nicholas A. Bellinger wrote:
> 
> 
> > So the idea of allowing the in-kernel CDB emulation to run after
> > user-space has returned unsupported opcode is problematic for a couple
> > of different reasons.
> > 
> > First, if the correct feature bits in standard INQUIRY + EVPD INQUIRY,
> > etc are not populated by user-space to match what in-kernel CDB
> > emulation actually supports, this can result in potentially undefined
> > results initiator side.
> > 
> > Also for example, if user-space decided to emulate only a subset of PR
> > operations, and leaves the rest of it up to the in-kernel emulation,
> > there's not really a way to sync current state between the two, which
> > also can end up with undefined results.
> > 
> > So that said, I think a saner approach would be two define two modes of
> > operation for TCMU:
> > 
> >*) Passthrough Mode: All CDBs are passed to user-space, and no
> >   in-kernel emulation is done in the event of an unsupported
> >   opcode response.
> > 
> >*) I/O Mode: I/O related CDBs are passed into user-space, but
> >   all control CDBs continue to be processed by in-kernel emulation.
> >   This effectively limits operation to TYPE_DISK, but with this mode
> >   it's probably OK to assume this.
> > 
> > This seems like the best trade-off between flexibility when everything
> > should be handled by user-space, vs. functionality when only block
> > remapping of I/O is occurring in user-space code.
> 
> The problem there is that the first case has all the issues of pscsi and 
> simply becomes a performance optimization over tgt+iscsi client+pscsi and 
> the latter case excludes the main use cases I'm interested in - OSDs, media 
> changers, optical discs (the biggest thing for me), and so forth.
> 
> One of the main things I want to do with this is hook up a plugin that uses 
> libmirage to handle various optical disc image formats.
> 

Not sure I follow..  How does the proposed passthrough mode prevent
someone from emulating OSDs, media changers, optical disks or anything
else in userspace with TCMU..?

The main thing that the above comments highlight is why attempting to
combine the existing in-kernel emulation with a userspace backend
providing it's own emulation can open up a number of problems with
mismatched state between the two.

--nab

--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 4/4] target: Add a user-passthrough backstore

2014-09-19 Thread Alex Elsayed
Nicholas A. Bellinger wrote:


> So the idea of allowing the in-kernel CDB emulation to run after
> user-space has returned unsupported opcode is problematic for a couple
> of different reasons.
> 
> First, if the correct feature bits in standard INQUIRY + EVPD INQUIRY,
> etc are not populated by user-space to match what in-kernel CDB
> emulation actually supports, this can result in potentially undefined
> results initiator side.
> 
> Also for example, if user-space decided to emulate only a subset of PR
> operations, and leaves the rest of it up to the in-kernel emulation,
> there's not really a way to sync current state between the two, which
> also can end up with undefined results.
> 
> So that said, I think a saner approach would be two define two modes of
> operation for TCMU:
> 
>*) Passthrough Mode: All CDBs are passed to user-space, and no
>   in-kernel emulation is done in the event of an unsupported
>   opcode response.
> 
>*) I/O Mode: I/O related CDBs are passed into user-space, but
>   all control CDBs continue to be processed by in-kernel emulation.
>   This effectively limits operation to TYPE_DISK, but with this mode
>   it's probably OK to assume this.
> 
> This seems like the best trade-off between flexibility when everything
> should be handled by user-space, vs. functionality when only block
> remapping of I/O is occurring in user-space code.

The problem there is that the first case has all the issues of pscsi and 
simply becomes a performance optimization over tgt+iscsi client+pscsi and 
the latter case excludes the main use cases I'm interested in - OSDs, media 
changers, optical discs (the biggest thing for me), and so forth.

One of the main things I want to do with this is hook up a plugin that uses 
libmirage to handle various optical disc image formats.

--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 4/4] target: Add a user-passthrough backstore

2014-09-19 Thread Nicholas A. Bellinger
Hi Andy,

A few comments are inline below.

On Mon, 2014-09-15 at 16:12 -0700, Andy Grover wrote:
> Add a LIO storage engine that presents commands to userspace for execution.
> This would allow more complex backstores to be implemented out-of-kernel,
> and also make experimentation a-la FUSE (but at the SCSI level -- "SUSE"?)
> possible.
> 
> It uses a mmap()able UIO device per LUN to share a command ring and data
> area. The commands are raw SCSI CDBs and iovs for in/out data. The command
> ring is also reused for returning scsi command status and optional sense
> data.
> 
> This implementation is based on Shaohua Li's earlier version but heavily
> modified. Differences include:
> 
> * Shared memory allocated by kernel, not locked-down user pages
> * Single ring for command request and response
> * Offsets instead of embedded pointers
> * Generic SCSI CDB passthrough instead of per-cmd specialization in ring
>   format.
> * Uses UIO device instead of anon_file passed in mailbox.
> * Optional in-kernel handling of some commands.
> 

Shaohua, do you have any comments are these changes..?  It's not clear
that all of these changes are beneficial, so I'd really like to hear
input given your the original author of this code.

> The main reason for these differences is to permit greater resiliency
> if the user process dies or hangs.
> 
> Things not yet implemented (on purpose):
> 
> * Zero copy. The data area is flexible enough to allow page flipping or
>   backend-allocated pages to be used by fabrics, but it's not clear these
>   are performance wins. Can come later.
> * Out-of-order command completion by userspace. Possible to add by just
>   allowing userspace to change cmd_id in rsp cmd entries, but currently
>   not supported.
> * No locks between kernel cmd submission and completion routines. Sounds
>   like it's possible, but this can come later.
> * Sparse allocation of mmaped area. Current code vmallocs the whole thing.
>   If the mapped area was larger and not fully mapped then the driver would
>   have more freedom to change cmd and data area sizes based on demand.
> 
> Current code open issues:
> 
> * The use of idrs may be overkill -- we maybe can replace them with a
>   simple counter to generate cmd_ids, and a hash table to get a cmd_id's
>   associated pointer.
> * Use of a free-running counter for cmd ring instead of explicit modulo
>   math. This would require power-of-2 cmd ring size.
> 
> Signed-off-by: Andy Grover 
> ---
>  drivers/target/Kconfig |5 +
>  drivers/target/Makefile|1 +
>  drivers/target/target_core_transport.c |4 +
>  drivers/target/target_core_user.c  | 1169 
> 
>  include/uapi/linux/Kbuild  |1 +
>  include/uapi/linux/target_core_user.h  |  142 
>  6 files changed, 1322 insertions(+)
>  create mode 100644 drivers/target/target_core_user.c
>  create mode 100644 include/uapi/linux/target_core_user.h
> 



> +
> +/* Core requires execute_rw be set, but just return unsupported */
> +static sense_reason_t
> +tcmu_retry_rw(struct se_cmd *cmd, struct scatterlist *sgl, u32 sgl_nents,
> +   enum dma_data_direction data_direction)
> +{
> + return TCM_UNSUPPORTED_SCSI_OPCODE;
> +}
> +
> +static struct sbc_ops tcmu_retry_ops = {
> + .execute_rw = tcmu_retry_rw,
> +};
> +
> +static void tcmu_work(struct work_struct *work)
> +{
> + struct tcmu_cmd *cmd = container_of(work, struct tcmu_cmd, work);
> + struct se_cmd *se_cmd = cmd->se_cmd;
> +
> + target_execute_cmd(se_cmd);
> + kmem_cache_free(tcmu_cmd_cache, cmd);
> +}
> +
> +static void tcmu_emulate_cmd(struct tcmu_cmd *cmd)
> +{
> + struct se_cmd *se_cmd = cmd->se_cmd;
> + sense_reason_t ret;
> + unsigned long flags;
> +
> + /* Re-run parsing to set execute_cmd to value for possible emulation */
> + se_cmd->execute_cmd = NULL;
> +
> + /*
> +  * Can't optionally call generic_request_failure if flags indicate it's
> +  * still being handled by us.
> + */
> + spin_lock_irqsave(&se_cmd->t_state_lock, flags);
> + se_cmd->transport_state &= ~CMD_T_BUSY|CMD_T_SENT;
> + spin_unlock_irqrestore(&se_cmd->t_state_lock, flags);
> +
> + ret = sbc_parse_cdb(se_cmd, &tcmu_retry_ops);
> + if (ret == TCM_NO_SENSE && se_cmd->execute_cmd) {
> + schedule_work(&cmd->work);
> + } else {
> + /* Can't emulate. */
> + transport_generic_request_failure(se_cmd, ret);
> + kmem_cache_free(tcmu_cmd_cache, cmd);
> + }
> +}

So the idea of allowing the in-kernel CDB emulation to run after
user-space has returned unsupported opcode is problematic for a couple
of different reasons.

First, if the correct feature bits in standard INQUIRY + EVPD INQUIRY,
etc are not populated by user-space to match what in-kernel CDB
emulation actually supports, this can result in potentially undefined
results initiator side.

Also for exa

[PATCH 4/4] target: Add a user-passthrough backstore

2014-09-15 Thread Andy Grover
Add a LIO storage engine that presents commands to userspace for execution.
This would allow more complex backstores to be implemented out-of-kernel,
and also make experimentation a-la FUSE (but at the SCSI level -- "SUSE"?)
possible.

It uses a mmap()able UIO device per LUN to share a command ring and data
area. The commands are raw SCSI CDBs and iovs for in/out data. The command
ring is also reused for returning scsi command status and optional sense
data.

This implementation is based on Shaohua Li's earlier version but heavily
modified. Differences include:

* Shared memory allocated by kernel, not locked-down user pages
* Single ring for command request and response
* Offsets instead of embedded pointers
* Generic SCSI CDB passthrough instead of per-cmd specialization in ring
  format.
* Uses UIO device instead of anon_file passed in mailbox.
* Optional in-kernel handling of some commands.

The main reason for these differences is to permit greater resiliency
if the user process dies or hangs.

Things not yet implemented (on purpose):

* Zero copy. The data area is flexible enough to allow page flipping or
  backend-allocated pages to be used by fabrics, but it's not clear these
  are performance wins. Can come later.
* Out-of-order command completion by userspace. Possible to add by just
  allowing userspace to change cmd_id in rsp cmd entries, but currently
  not supported.
* No locks between kernel cmd submission and completion routines. Sounds
  like it's possible, but this can come later.
* Sparse allocation of mmaped area. Current code vmallocs the whole thing.
  If the mapped area was larger and not fully mapped then the driver would
  have more freedom to change cmd and data area sizes based on demand.

Current code open issues:

* The use of idrs may be overkill -- we maybe can replace them with a
  simple counter to generate cmd_ids, and a hash table to get a cmd_id's
  associated pointer.
* Use of a free-running counter for cmd ring instead of explicit modulo
  math. This would require power-of-2 cmd ring size.

Signed-off-by: Andy Grover 
---
 drivers/target/Kconfig |5 +
 drivers/target/Makefile|1 +
 drivers/target/target_core_transport.c |4 +
 drivers/target/target_core_user.c  | 1169 
 include/uapi/linux/Kbuild  |1 +
 include/uapi/linux/target_core_user.h  |  142 
 6 files changed, 1322 insertions(+)
 create mode 100644 drivers/target/target_core_user.c
 create mode 100644 include/uapi/linux/target_core_user.h

diff --git a/drivers/target/Kconfig b/drivers/target/Kconfig
index dc2d84a..b03a845 100644
--- a/drivers/target/Kconfig
+++ b/drivers/target/Kconfig
@@ -31,6 +31,11 @@ config TCM_PSCSI
Say Y here to enable the TCM/pSCSI subsystem plugin for non-buffered
passthrough access to Linux/SCSI device
 
+config TCM_USER
+   tristate "TCM/USER Subsystem Plugin for Linux"
+   help
+   Say Y here to enable the TCM/USER subsystem plugin
+
 source "drivers/target/loopback/Kconfig"
 source "drivers/target/tcm_fc/Kconfig"
 source "drivers/target/iscsi/Kconfig"
diff --git a/drivers/target/Makefile b/drivers/target/Makefile
index 85b012d..bbb4a7d 100644
--- a/drivers/target/Makefile
+++ b/drivers/target/Makefile
@@ -22,6 +22,7 @@ obj-$(CONFIG_TARGET_CORE) += target_core_mod.o
 obj-$(CONFIG_TCM_IBLOCK)   += target_core_iblock.o
 obj-$(CONFIG_TCM_FILEIO)   += target_core_file.o
 obj-$(CONFIG_TCM_PSCSI)+= target_core_pscsi.o
+obj-$(CONFIG_TCM_USER) += target_core_user.o
 
 # Fabric modules
 obj-$(CONFIG_LOOPBACK_TARGET)  += loopback/
diff --git a/drivers/target/target_core_transport.c 
b/drivers/target/target_core_transport.c
index 7fa62fc..f018a8c 100644
--- a/drivers/target/target_core_transport.c
+++ b/drivers/target/target_core_transport.c
@@ -232,6 +232,10 @@ void transport_subsystem_check_init(void)
if (ret != 0)
pr_err("Unable to load target_core_pscsi\n");
 
+   ret = request_module("target_core_user");
+   if (ret != 0)
+   pr_err("Unable to load target_core_user\n");
+
sub_api_initialized = 1;
 }
 
diff --git a/drivers/target/target_core_user.c 
b/drivers/target/target_core_user.c
new file mode 100644
index 000..33b56a8
--- /dev/null
+++ b/drivers/target/target_core_user.c
@@ -0,0 +1,1169 @@
+/*
+ * Copyright (C) 2013 Shaohua Li 
+ * Copyright (C) 2014 Red Hat, Inc.
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ * You should have received a copy of the GNU General