tcm_loop

The Linux SCSI Target Wiki

(Difference between revisions)
Jump to: navigation, search
m (See also)
m
 
(120 intermediate revisions not shown)
Line 1: Line 1:
-
'''tcm_loop''' is a fabric module that provides full local SCSI device emulation ([[SPC-3]] and [[SPC-4]]) for raw block storage objects, from both kernel and user space.
+
{{DISPLAYTITLE:tcm_loop}}
-
{{AdSense right}}
+
-
 
+
-
== Overview ==
+
-
 
+
{{Infobox software
{{Infobox software
-
|name                       = LIO Target
+
| name                   = {{Target}}
-
|logo                       = [[Image:RisingTide_Logo_small.png|180px|Logo]]
+
| logo                   = [[Image:Corp_Logo.png|180px|Logo]]
-
|screenshot                 = RisingTide Systems
+
| screenshot             = {{RTS screenshot|tcm_loop}}
-
|caption                   = tcm_loop SCSI Emulation Device
+
| caption               = tcm_loop SCSI Emulation Device
-
|collapsible               =  
+
| collapsible           =  
-
|author                     = Nicholas Bellinger
+
| author                 = {{Nicholas Bellinger}}
-
|developer                 = RisingTide Systems LLC
+
| developer             = {{RTS legal}}
-
|released                   = {{RTS releases|tcm_loop|initial_date}}
+
| released               = {{RTS releases|tcm_loop|initial_date}}
-
|discontinued               = {{RTS releases|tcm_loop|eol_date}}
+
| discontinued           = {{RTS releases|tcm_loop|eol_date}}
-
|latest release version     = {{RTS releases|tcm_loop|release_ver}}
+
| latest release version = {{RTS releases|tcm_loop|release_ver}}
-
|latest release date       = {{RTS releases|tcm_loop|release_date}}
+
| latest release date   = {{RTS releases|tcm_loop|release_date}}
-
|latest preview version     = {{RTS releases|tcm_loop|preview_ver}}
+
| latest preview version = {{RTS releases|tcm_loop|preview_ver}}
-
|latest preview date       = {{RTS releases|tcm_loop|preview_date}}
+
| latest preview date   = {{RTS releases|tcm_loop|preview_date}}
-
|frequently updated         =
+
| status                = {{RTS releases|tcm_loop|status}}
-
|programming language       = C
+
| frequently updated     =
-
|operating system           = Linux
+
| programming language   = C
-
|platform                   =
+
| operating system       = Linux
-
|size                       =
+
| platform               =
-
|language                   =
+
| size                   =
-
|status                    = Production
+
| language               =
-
|genre                     = Fabric module
+
| genre                 = Fabric module
-
|license                   = GNU General Public License
+
| license               = {{GPLv2}}
-
|website                   = {{RTS website}}
+
| website               = {{RTS website}}
}}
}}
 +
:''See [[LIO]] for a complete overview over all fabric modules.''
-
{{Image|TCM-KVM-Megasas-8708EM2-Windows7-x64.png|[[FILEIO]] with a tcm_loop SAS port on KVM x86_64 host running v2.6.34 with the new LSI Megasas 8708EM2 PPC HBA emulation into a Windows 7 x64 KVM Guest.}}
+
{{Image|TCM-KVM-Megasas-8708EM2-Windows7-x64.png|[[FILEIO]] with a tcm_loop SAS port on [[Kernel-based Virtual Machine|KVM]] x86_64 host running v2.6.34 with the new LSI Megasas 8708EM2 PPC HBA emulation into a Windows 7 x64 [[Kernel-based Virtual Machine|KVM]] Guest.}}
 +
<!--
 +
{{Image|TCM Loop-OS2Warp-QEMU-KVM-HPFS.png|[[FILEIO]] with a tcm_loop SCSI port on [[Kernel-based Virtual Machine|KVM]] x86_64 host running v2.6.36-rc3+ with [[QEMU]] lsi53c895a HBA emulation into an OS/2 Warp v4 guest. A HPFS partition has been formatted and mounted as DRIVE D:.}}
 +
{{Image|TCM_Loop-3.2-RHEL5-NetBSD5.png|tcm_loop fabric module running on {{T}} providing shared virtual SAS target emulation into a VM hypervisor to multiple VM guests of RHEL 5.4 x86_64 and NetBSD 5.x 32-bit.}}
 +
{{Image|TCM Loop-3.2-RHEL5-vmw6.png|tcm_loop fabric module running on {{T}} v3.2 on Linux Host v2.6.32-rc5 providing persistent reservations to RHEL 5.4 x86_64 [[Kernel-based Virtual Machine|KVM]] Guest using scsi generic passthrough with VMware Workstation 6.5.}}
 +
-->
-
{{Image|TCM Loop-OS2Warp-QEMU-KVM-HPFS.png|[[FILEIO]] with a tcm_loop SCSI port on KVM x86_64 host running v2.6.36-rc3+ with QEMU lsi53c895a HBA emulation into an OS/2 Warp v4 guest. A HPFS partition has been formatted and mounted as DRIVE D:}}
+
'''tcm_loop''' is a fabric module that can emulate any supported fabric type locally for {{Target}}.
-
{{Image|TCM_Loop-3.2-RHEL5-NetBSD5.png|<code>tcm_loop</code> fabric module running on LIO v3.2 on a Linux Host providing shared virtual SAS Target emulation into a VM hypervisor to multiple VM guests of RHEL 5.4 x86_64 and NetBSD 5.x 32-bit.}}
+
== Overview ==
-
{{Image|TCM Loop-3.2-RHEL5-vmw6.png|<code>tcm_loop</code> fabric module running on LIO v3.2 on Linux Host v2.6.32-rc5 providing persistent reservations to RHEL 5.4 x86_64 KVM Guest using scsi generic passthrough with Vmware Workstation 6.5.}}
+
tcm_loop provides full local SCSI device emulation ([[SPC-3]] and [[SPC-4]]) for raw block storage objects. It thus makes any raw block device apear to the {{Target}} as a full [[SPC-3]]/[[SPC-4]] [[SCSI]] compliant device with the corresponding [[CDB]]-level SAS emulation, for access from any SCSI LLD driver.
-
'''tcm_loop''' is a pluggable virtual fabric module ([http://git.kernel.org/?p=linux/kernel/git/nab/lio-core-2.6.git;a=tree;f=drivers/target/tcm_loop;hb=HEAD tcm_loop.ko]) for the LIO multiprotocol [[Target]] engine.  It makes any raw block device apear to the [[Target]] as a full [[SPC-3]]/[[SPC-4]] [[SCSI]] compliant device with the corresponding [[CDB]]-level SAS emulation, for access from any SCSI LLD driver.
+
tcm_loop minimizes overhead - it delivers >12&nbsp;GB/s (native) and >2&nbsp;GB/s (from a single [[Kernel-based Virtual Machine|KVM]] guest, without [http://en.wikipedia.org/wiki/Hardware-assisted_virtualization hardware assist]) on Intel [http://en.wikipedia.org/wiki/Nehalem_(microarchitecture) Nehalem] platforms. This provides e.g., unmodified guest operating systems, such as [[VMware ESX]] or [[VMware vSphere]], to use the full [[SPC-3]]/[[SPC-4]] [[SCSI]] functionality on any type of local raw block storage object they require.
-
tcm_loop minimizes overhead: it delivers >12&nbsp;GB/s (native) and >2&nbsp;GB/s (from a single [[KVM]] guest, without hardware assist) on Intel Nehalem platforms. This provides e.g., unmodified guest operating systems, such as [[VMware ESX]] or [[VMware vSphere]] to use the full [[SPC-4]] SCSI functionality on any type of local raw block storage object they require.
+
The tcm_loop fabric module ({{RTS releases|tcm_loop|module_repo}}, {{RTS releases|tcm_loop|module_info}}) for {{T}} was released with Linux kernel {{RTS releases|tcm_loop|kernel_ver}} on {{RTS releases|tcm_loop|initial_date}}.<ref>{{RTS releases|tcm_loop|kernel_rel}}</ref>
-
The initial release of tcm_loop was [http://git.kernel.org/?p=linux/kernel/git/nab/lio-core-2.6.git;a=commitdiff;h=10eae38203a01a26fca3b2097f13e48a0ba2d38f announced] on 10/19/2009.
+
== targetcli ==
-
== Specification compliance ==
+
''[[targetcli]]'' from {{RTS full}} is used to configure tcm_loop targets. ''targetcli'' aggregates {{T}} service modules via a core library, and exports them through an API, to provide a unified single-node SAN configuration shell, independently of the underlying fabric(s).
-
The type of SCSI fabric features that are available with tcm_loop (Target mode emulated SAS ports) includes the complete set of SPC-3 logic in the [[Target]] engine including advanced features such as [[Persistent_Reservations|PRs]], [[ALUA]], etc.
+
{{Ambox| type=info| head=[[LIO]] Admin Manual| text=The {{LIO Admin Manual}} provides comprehensive background and many examples on using ''targetcli'' and on programming the {{RTS short}} library.}}
-
tcm_loop now (v4) also supports ''high level fabric'' emulation. That is, depending on which [[WWN]] is used to create the configFS group in <code>/sys/kernel/config/target/loopback/$WWN</code>, it configures itself to return iSCSI, SAS or FC device identifiers for the SCSI LUN protocol identifiers in the EVPD 0x83 device identifier page.
+
=== Startup ===
-
The entire corresponding logic resides in <code>target_core_fabric_lib.c</code>, and is shared by the different target fabric modules. The code simply contains magic numbers defined for different protocols in [[SPC-4]], and is commented using pieces directly from [[SPC-4]].
+
[[targetcli]] is invoked by running ''targetcli'' as root from the command prompt of the underlying {{OS}} shell. An initial setup as described in targetcli is assumed.
-
== CLI ==
+
<pre>
 +
# targetcli
 +
Welcome to targetcli:
-
The [http://linux-iscsi.org/builds/docs/LIO-3.0-Users_Reference_Manual.pdf LIO 3.0 CLI] manual describes a simple userspace shell ([[lio-utils]]) for LIO v3.0 ([[Target]]+[[iSCSI]]). It also describes how to setup a number of different subsystem storage object backstores for LIO LUN endpoints:
+
Copyright (c) 2014 by Datera, Inc.
 +
All rights reserved.
-
* [[Lio-utils#Target|Target CLI]]: basic Target v3.x CLI operations using tcm_node.py from [[lio-utils]].
+
Visit us at http://www.datera.io.
-
* [[Lio-utils#iSCSI|iSCSI CLI]]: basic [[iSCSI]] v3.x CLI operations using lio_node.py and lio_dump.py from [[lio-utils]].
+
-
[[RisingTide Systems]] makes available upon request and for free [[RTSadmin]], a comprehensive, powerful, yet easy storage management tool that can easily handle even complex LIO installations.
+
Using ib_srpt fabric module.
 +
Using qla2xxx fabric module.
 +
Using iscsi fabric module.
 +
Using loopback fabric module.
-
== Example ==
+
/> loopback/ info
-
 
+
Fabric module name: loopback
-
Here is an example of creating two local virtual SCSI devices from a single [[IBLOCK]] object on two virtual SAS Target port addresses, which are prefixed here with "naa."
+
ConfigFS path: /sys/kernel/config/target/loopback
 +
Allowed WWN list type: naa
 +
Fabric module specfile: /var/target/fabric/loopback.spec
 +
Fabric module features: nexus
 +
Corresponding kernel module: tcm_loop
 +
/>
 +
</pre>
-
The simple userspace shell ([[lio-utils]]) for LIO v3.0 configuration was used in this example ([[RTSadmin]] is a better management tool).
+
=== Cheat sheet ===
-
=== Setup ===
+
{| class="table-left"
 +
|- bgcolor=#a0c0f0
 +
! Command
 +
! Comment
 +
|-
 +
| ''/backstores/iblock create my_disk /dev/sdb''
 +
| Create the LUN ''my_disk'' on the device ''/dev/sdb''
 +
|-
 +
| In /backstores/iblock/my_disk>:<br/>''/loopback create''
 +
| Create a local loopback target
 +
|-
 +
| ''/saveconfig''
 +
| Commit the configuration
 +
|-
 +
|}
-
==== Create first I_T Nexus ====
+
=== Display the object tree ===
-
Create the first I_T Nexus that creates a new Linux/SCSI <code>struct Scsi_Host</code>:
+
Use ''ls'' to list the object hierarchy, which is initially empty:
<pre>
<pre>
-
  target:/usr/src/lio-utils.git# tcm_loop --createnexus 1
+
/> ls
-
  Successfully created virtual SCSI I_T Nexus between TCM and Linux/SCSI HBA
+
o- / ..................................................................... [...]
-
    SAS Target Address: naa.600140554cf3a18e
+
  o- backstores .......................................................... [...]
-
    SAS Initiator Address naa.60014053226f0388
+
  | o- fileio ............................................... [0 Storage Object]
 +
  | o- iblock ............................................... [0 Storage Object]
 +
  | o- pscsi ................................................ [0 Storage Object]
 +
  | o- rd_dr ................................................ [0 Storage Object]
 +
  | o- rd_mcp ............................................... [0 Storage Object]
 +
  o- ib_srpt ........................................................ [0 Target]
 +
  o- iscsi .......................................................... [0 Target]
 +
  o- loopback ....................................................... [0 Target]
 +
  o- qla2xxx ........................................................ [0 Target]
 +
/>
</pre>
</pre>
-
==== Create LUN ====
+
{{Ambox| type=info| head=Global parameter ''auto_cd_after_create''| text=After the creation of a new object, automatically enter its object context.}}
-
Create the LUN on the SAS Target port and rescan the Linux SCSI LLD by SCSI HCTL bus address:
+
Per default, ''auto_cd_after_create'' is set to ''true'', which automatically enters an object context (or working directory) after its creation. The examples here are modeled after this behavior.
 +
 
 +
Optionally, set ''auto_cd_after_create=false'' to prevent ''targetcli'' from automatically entering new object contexts after their creation:
<pre>
<pre>
-
  target:/usr/src/lio-utils.git# tcm_loop --addlun naa.600140554cf3a18e 1 0 iblock_0/lvm_test0
+
/> set global auto_cd_after_create=false
-
  Successfully created SAS Target Port to local virtual SCSI Logical Unit
+
Parameter auto_cd_after_create is now 'false'.
 +
/>
</pre>
</pre>
-
==== Create second I_T Nexus ====
+
=== Create a backstore ===
 +
 
 +
Create a backstore using the IBLOCK or FILEIO type devices.
-
Create the second I_T Nexus that creates another virtual Linux SCSI HBA:
+
For instance, enter the top-level ''backstore'' context and create an IBLOCK backstore from a ''/dev/sdb'' block device:
<pre>
<pre>
-
  target:/usr/src/lio-utils.git# tcm_loop --createnexus 1
+
/> cd backstores/
-
  Successfully created virtual SCSI I_T Nexus between TCM and Linux/SCSI HBA
+
/backstores> iblock/ create name=my_disk dev=/dev/sdb
-
    SAS Target Address: naa.600140566ce31347
+
Generating a wwn serial.
-
    SAS Initiator Address naa.6001405fd3afa2ef
+
Created iblock storage object my_disk using /dev/sdb.
-
  target:/usr/src/lio-utils.git# tcm_loop --addlun naa.600140566ce31347 1 0 iblock_0/lvm_test0
+
Entering new node /backstores/iblock/my_disk.
-
  Successfully created SAS Target Port to local virtual SCSI Logical Unit
+
/backstores/iblock/my_disk>
</pre>
</pre>
-
=== Results ===
+
''targetcli'' automatically creates a WWN serial ID for the backstore device and then changes the working context to it.
-
Healthy results should look similar to the following examples, which are provided as a reference.
+
The resulting object hierarchy looks as follows (displayed from the root object):
-
 
+
-
==== ConfigFS object tree ====
+
-
 
+
-
The resulting [[Target/configFS]] object tree in <code>/sys/kernel/config/target/loopback/</code>:
+
<pre>
<pre>
-
  /sys/kernel/config/target/loopback/
+
/> ls
-
  |-- naa.600140554cf3a18e
+
o- / ..................................................................... [...]
-
   |  `-- tpgt_1
+
   o- backstores .......................................................... [...]
-
   |       |-- lun
+
   | o- fileio ............................................... [0 Storage Object]
-
   |       |  `-- lun_0
+
   | o- iblock ............................................... [1 Storage Object]
-
   |       |       `-- virtual_scsi_port -> ../../../../../../target/core/iblock_0/lvm_test0
+
   | | o- my_disk .......................................... [/dev/sdb activated]
-
   |       `-- nexus
+
   | o- pscsi ................................................ [0 Storage Object]
-
   |           `-- naa.60014053226f0388
+
   | o- rd_dr ................................................ [0 Storage Object]
-
   |-- naa.600140566ce31347
+
   | o- rd_mcp ............................................... [0 Storage Object]
-
   |  `-- tpgt_1
+
   o- ib_srpt ........................................................ [0 Target]
-
   |      |-- lun
+
   o- iscsi .......................................................... [0 Target]
-
   |      |  `-- lun_0
+
   o- loopback ....................................................... [0 Target]
-
  |      |      `-- virtual_scsi_port -> ../../../../../../target/core/iblock_0/lvm_test0
+
   o- qla2xxx ........................................................ [0 Target]
-
   |      `-- nexus
+
/>
-
  |          `-- naa.6001405fd3afa2ef
+
-
  `-- version
+
</pre>
</pre>
-
==== SCSI devices ====
+
Alternatively, any LVM logical volume can be used as a backstore, please refer to the {{LIO Admin Manual}} on how to create them properly.
-
The list of resulting SCSI devices with:
+
For instance, create an IBLOCK backstore on a logical volume (under ''/dev/<volume_group_name>/<logical_volume_name>''):
<pre>
<pre>
-
  lsscsi -v output
+
/backstores> iblock/ create name=my_disk dev=/dev/vg0/lv1
 +
Generating a wwn serial.
 +
Created iblock storage object my_disk using /dev/vg0/lv1.
 +
Entering new node /backstores/iblock/my_disk.
 +
/backstores/iblock/my_disk>
</pre>
</pre>
-
is:
+
Again, ''targetcli'' automatically creates a WWN serial ID for the backstore device and then changes the working context to it.
-
<small><pre>
+
{{Ambox| type=info| head=More backstore examples| text=More examples on creating backstores can be found in ''[[targetcli#Create a backstore|targetcli]]''.}}
-
  [9:0:1:0]    disk    LIO-ORG  IBLOCK          3.1  /dev/sdg
+
-
    dir: /sys/bus/scsi/devices/9:0:1:0  [/sys/devices/tcm_loop_0/tcm_loop_adapter_0/host9/target9:0:1/9:0:1:0]
+
-
  [10:0:1:0]   disk    LIO-ORG  IBLOCK          3.1  /dev/sdh
+
-
    dir: /sys/bus/scsi/devices/10:0:1:0  [/sys/devices/tcm_loop_0/tcm_loop_adapter_1/host10/target10:0:1/10:0:1:0]
+
-
</pre></small>
+
-
==== Block devices ====
+
=== Instantiate a target ===
-
The list of resulting block devices with:
+
Instantiate a loopback target on the existing IBLOCK backstore device ''my_disk'' (as set up in [[targetcli]]):
<pre>
<pre>
-
  sg_inq -i /dev/sdg
+
/backstores/iblock/my_disk> /loopback create
 +
Successfully created LUN 0.
 +
Entering new node /loopback/naa.60014057f822d991/luns/lun0.
 +
/loopback/naa...991/luns/lun0>
</pre>
</pre>
-
is:
+
''targetcli'' automatically assigns the default ID '0' to the LUN, and then changes the working context to the new LUN. Now the target is created, and exports ''/dev/sdb'' as LUN 0.
 +
 
 +
Return to the root context, as no attributes need to be set or modified for a standard loopback target:
<pre>
<pre>
-
  VPD INQUIRY: Device Identification page
+
/loopback/naa...991/luns/lun0> cd /
-
  Designation descriptor number 1, descriptor length: 20
+
/>
-
    id_type: NAA,  code_set: Binary
+
-
    associated with the addressed logical unit
+
-
      NAA 6, IEEE Company_id: 0x1405
+
-
      Vendor Specific Identifier: 0xa97e4ce21
+
-
      Vendor Specific Identifier Extension: 0xc0711de829b000c2
+
-
      [0x6001405a97e4ce21c0711de829b000c2]
+
-
  Designation descriptor number 2, descriptor length: 52
+
-
    id_type: T10 vendor identification,  code_set: ASCII
+
-
    associated with the addressed logical unit
+
-
      vendor id: LIO-ORG
+
-
      vendor specific: IBLOCK:a97e4ce21c0711de829b000c2943d57b
+
-
  Designation descriptor number 3, descriptor length: 8
+
-
    transport: Serial Attached SCSI (SAS)
+
-
    id_type: Relative target port,  code_set: Binary
+
-
    associated with the target port
+
-
      Relative target port: 0x8
+
-
  Designation descriptor number 4, descriptor length: 8
+
-
    transport: Serial Attached SCSI (SAS)
+
-
    id_type: Target port group,  code_set: Binary
+
-
    associated with the target port
+
-
      Target port group: 0x0
+
-
  Designation descriptor number 5, descriptor length: 8
+
-
    id_type: Logical unit group,  code_set: Binary
+
-
    associated with the addressed logical unit
+
-
      Logical unit group: 0x0
+
-
  Designation descriptor number 6, descriptor length: 36
+
-
    transport: Serial Attached SCSI (SAS)
+
-
    id_type: SCSI name string,  code_set: UTF-8
+
-
    associated with the target port
+
-
      SCSI name string:
+
-
      naa.600140554cf3a18e,t,0x0001
+
</pre>
</pre>
-
==== Modules ====
+
=== Display the object tree ===
-
The resulting loadable kernel modules (LKMs) with:
+
The resulting object hierarchy looks as follows (displayed from the root object):
<pre>
<pre>
-
   lsmod output
+
/> ls
 +
o- / ..................................................................... [...]
 +
   o- backstores .......................................................... [...]
 +
  | o- fileio ............................................... [0 Storage Object]
 +
  | o- iblock ............................................... [1 Storage Object]
 +
  | | o- my_disk .......................................... [/dev/sdb activated]
 +
  | o- pscsi ................................................ [0 Storage Object]
 +
  | o- rd_dr ................................................ [0 Storage Object]
 +
  | o- rd_mcp ............................................... [0 Storage Object]
 +
  o- ib_srpt ........................................................ [0 Target]
 +
  o- iscsi .......................................................... [0 Target]
 +
  o- loopback ....................................................... [1 Target]
 +
  | o- naa.60014057f822d991 ............................. [naa.60014053485b9490]
 +
  | | o- luns .......................................................... [1 LUN]
 +
  | |  o- lun0 .................................... [iblock/my_disk (/dev/sbd)]
 +
  o- qla2xxx ........................................................ [0 Target]
 +
/>
</pre>
</pre>
-
are as follows:
+
=== Persist the configuration ===
 +
 
 +
{{Ambox| type=warning| head=Don't forget to use ''saveconfig''!| text=Without ''saveconfig'', the {{T}} configuration will be lost upon rebooting or unloading the target service, as it will revert back to the last saved one.}}
 +
 
 +
Use ''saveconfig'' from the root context to persist the {{T}} configuration across OS reboots:
<pre>
<pre>
-
  Module                  Size  Used by
+
/> saveconfig
-
  tcm_loop              11606  9
+
WARNING: Saving rtsnode1 current configuration to disk will overwrite your boot settings.
-
  iscsi_target_mod      205640  44
+
The current target configuration will become the default boot config.
-
  target_core_mod      226770  75 tcm_loop,iscsi_target_mod
+
Are you sure? Type 'yes': yes
-
  scsi_tgt                8806  1 target_core_mod
+
Making backup of srpt/ConfigFS with timestamp: 2012-02-27_23:19:37.660264
-
  configfs              21168  4 tcm_loop,iscsi_target_mod,target_core_mod
+
Successfully updated default config /etc/target/srpt_start.sh
-
  dm_mirror              10356  0
+
Making backup of qla2xxx/ConfigFS with timestamp: 2012-02-27_23:19:37.660264
-
  dm_region_hash          7562  1 dm_mirror
+
Successfully updated default config /etc/target/qla2xxx_start.sh
-
  dm_log                  6769  2 dm_mirror,dm_region_hash
+
Making backup of loopback/ConfigFS with timestamp: 2012-02-27_23:19:37.660264
-
  dm_snapshot            18562  0
+
Successfully updated default config /etc/target/loopback_start.sh
-
  pcnet32                25510  0
+
Making backup of LIO-Target/ConfigFS with timestamp: 2012-02-27_23:19:37.660264
-
  mii                    3430  1 pcnet32
+
Successfully updated default config /etc/target/lio_backup-2012-02-27_23:19:37.660264.sh
 +
Making backup of Target_Core_Mod/ConfigFS with timestamp: 2012-02-27_23:19:37.660264
 +
Successfully updated default config /etc/target/tcm_backup-2012-02-27_23:19:37.660264.sh
 +
Generated Target_Core_Mod config: /etc/target/backup/tcm_backup-2012-02-27_23:19:37.660264.sh
 +
Successfully updated default config /etc/target/lio_start.sh
 +
Successfully updated default config /etc/target/tcm_start.sh
 +
/>
</pre>
</pre>
-
==== Kernel ringbuffer ====
+
=== Spec file ===
-
The resulting messages in the kernel ringbuffer, dumped with ''dmesg'', should look similar to:
+
{{RTS short}} spec files define the fabric-dependent feature set, capabilities and available target ports of the specific underlying fabric. In particular, the tcm_loop spec file ''/var/target/fabric/loopback.spec'' is included via RTSlib.
-
<small><pre>
+
<pre>
-
  Target_Core_ConfigFS: REGISTER -> group: e1193d80 name: loopback
+
# WARNING: This is a draft specfile supplied for demo purposes only.
-
  Initialized TCM Loop Core Bus
+
-
  <<<<<<<<<<<<<<<<<<<<<< BEGIN FABRIC API >>>>>>>>>>>>>>>>>>>>>>
+
-
  Initialized struct target_fabric_configfs: df226208 for loopback
+
-
  <<<<<<<<<<<<<<<<<<<<<< END FABRIC API >>>>>>>>>>>>>>>>>>>>>>
+
-
  TCM_LOOP[0] - Set fabric -> tcm_loop_fabric_configfs
+
-
  Target_Core_ConfigFS: REGISTER -> Located fabric: loopback
+
-
  Target_Core_ConfigFS: REGISTER -> e17bcddc
+
-
  Target_Core_ConfigFS: REGISTER -> Allocated Fabric: loopback
+
-
  Target_Core_ConfigFS: REGISTER -> Set tf->tf_fabric for loopback
+
-
  scsi9 : TCM_Loopback
+
-
  TCM_Loop_ConfigFS: Allocated emulated Target SAS Address: naa.600140554cf3a18e at Linux/SCSI Host ID: 9
+
-
  TARGET_CORE[loopback]: Allocated Normal se_portal_group_t for endpoint: naa.600140554cf3a18e, Portal Tag: 1
+
-
  TCM_Loop_ConfigFS: Allocated Emulated SAS Target Port naa.600140554cf3a18e,t,0x0001
+
-
  loopback_TPG[1] - Added DYNAMIC ACL with TCQ Depth: 1 for loopback Initiator Node: naa.60014053226f0388
+
-
  TARGET_CORE[loopback]: Registered fabric_sess_ptr: d43376e8
+
-
  TCM_Loop_ConfigFS: Established I_T Nexus to emulated SAS Initiator Port: naa.60014053226f0388
+
-
  iblock/loopback: Adding to default ALUA Target Port Group: alua/default_tg_pt_gp
+
-
  loopback_TPG[1]_LUN[0] - Activated loopback Logical Unit from CORE HBA: 3
+
-
  TARGET_CORE[loopback]->TPG[1]_LUN[0] - Adding READ-WRITE access for LUN in Demo Mode
+
-
  TCM_Loop_ConfigFS: Port Link Successful
+
-
  scsi 9:0:1:0: Direct-Access    LIO-ORG  IBLOCK          3.1  PQ: 0 ANSI: 5
+
-
  sd 9:0:1:0: Attached scsi generic sg7 type 0
+
-
  sd 9:0:1:0: [sdg] READ CAPACITY(16) failed
+
-
  sd 9:0:1:0: [sdg] Result: hostbyte=0x05 driverbyte=0x00
+
-
  sd 9:0:1:0: [sdg] Sense not available.
+
-
  sd 9:0:1:0: [sdg] 3072000 512-byte logical blocks: (1.57 GB/1.46 GiB)
+
-
  sd 9:0:1:0: [sdg] Write Protect is off
+
-
  sd 9:0:1:0: [sdg] Mode Sense: 2f 00 00 00
+
-
  sd 9:0:1:0: [sdg] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
+
-
  sd 9:0:1:0: [sdg] READ CAPACITY(16) failed
+
-
  sd 9:0:1:0: [sdg] Result: hostbyte=0x05 driverbyte=0x00
+
-
  sd 9:0:1:0: [sdg] Sense not available.
+
-
    sdg: unknown partition table
+
-
  sd 9:0:1:0: [sdg] READ CAPACITY(16) failed
+
-
  sd 9:0:1:0: [sdg] Result: hostbyte=0x05 driverbyte=0x00
+
-
  sd 9:0:1:0: [sdg] Sense not available.
+
-
  sd 9:0:1:0: [sdg] Attached SCSI disk
+
-
  scsi10 : TCM_Loopback
+
-
  TCM_Loop_ConfigFS: Allocated emulated Target SAS Address: naa.600140566ce31347 at Linux/SCSI Host ID: 10
+
-
  TARGET_CORE[loopback]: Allocated Normal se_portal_group_t for endpoint: naa.600140566ce31347, Portal Tag: 1
+
-
  TCM_Loop_ConfigFS: Allocated Emulated SAS Target Port naa.600140566ce31347,t,0x0001
+
-
  loopback_TPG[1] - Added DYNAMIC ACL with TCQ Depth: 1 for loopback Initiator Node: naa.6001405fd3afa2ef
+
-
  TARGET_CORE[loopback]: Registered fabric_sess_ptr: d4337a28
+
-
  TCM_Loop_ConfigFS: Established I_T Nexus to emulated SAS Initiator Port: naa.6001405fd3afa2ef
+
-
  iblock/loopback: Adding to default ALUA Target Port Group: alua/default_tg_pt_gp
+
-
  loopback_TPG[1]_LUN[0] - Activated loopback Logical Unit from CORE HBA: 3
+
-
  TARGET_CORE[loopback]->TPG[1]_LUN[0] - Adding READ-WRITE access for LUN in Demo Mode
+
-
  TCM_Loop_ConfigFS: Port Link Successful
+
-
  scsi 10:0:1:0: Direct-Access    LIO-ORG  IBLOCK          3.1  PQ: 0 ANSI: 5
+
-
  sd 10:0:1:0: Attached scsi generic sg8 type 0
+
-
  sd 10:0:1:0: [sdh] READ CAPACITY(16) failed
+
-
  sd 10:0:1:0: [sdh] Result: hostbyte=0x05 driverbyte=0x00
+
-
  sd 10:0:1:0: [sdh] Sense not available.
+
-
  sd 10:0:1:0: [sdh] 3072000 512-byte logical blocks: (1.57 GB/1.46 GiB)
+
-
  sd 10:0:1:0: [sdh] Write Protect is off
+
-
  sd 10:0:1:0: [sdh] Mode Sense: 2f 00 00 00
+
-
  sd 10:0:1:0: [sdh] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
+
-
  sd 10:0:1:0: [sdh] READ CAPACITY(16) failed
+
-
  sd 10:0:1:0: [sdh] Result: hostbyte=0x05 driverbyte=0x00
+
-
  sd 10:0:1:0: [sdh] Sense not available.
+
-
    sdh: unknown partition table
+
-
  sd 10:0:1:0: [sdh] READ CAPACITY(16) failed
+
-
  sd 10:0:1:0: [sdh] Result: hostbyte=0x05 driverbyte=0x00
+
-
  sd 10:0:1:0: [sdh] Sense not available.
+
-
  sd 10:0:1:0: [sdh] Attached SCSI disk
+
-
</pre></small>
+
-
== Timeline ==
+
# The loopback fabric module uses the default feature set
-
{{LIO Timeline}}
+
features = nexus
 +
 
 +
# Use naa WWNs
 +
wwn_type = naa
 +
 
 +
# Non-standard module naming scheme
 +
kernel_module = tcm_loop
 +
 
 +
# The configfs group
 +
configfs_group = loopback
 +
</pre>
 +
 
 +
== Specifications ==
 +
 
 +
The type of [[SCSI]] fabric features that are available with tcm_loop (target mode emulated SAS ports) includes the complete set of SPC-3/SPC-4 logic in the {{T}} engine, including advanced features such as [[PRs]], [[ALUA]], [[VAAI]], etc.
 +
 
 +
tcm_loop supports high level fabric emulation. That is, depending on which [[WWN]] is used to create the [[ConfigFS]] group in ''/sys/kernel/config/target/loopback/$WWN'', it configures itself to return iSCSI, FC etc. device identifiers for the SCSI LUN protocol identifiers device identifier page.
== See also ==
== See also ==
-
* [[Target]] / [[SCSI]]
+
* [[{{OS}}]]
-
* Other fabric modules: [[iSCSI]], [[Fibre Channel over Ethernet|FCoE]], [[Fibre Channel]] ([[QLogic]]) and [[InfiniBand]] (future release)
+
* {{Target}}, [[targetcli]]
-
* Platforms: PC architecture (x86/ia64), [[PlayStation 2]] and [[PlayStation 3]] (Cell), etc.
+
* [[FCoE]], [[Fibre Channel]], [[iSCSI]], [[iSER]], [[SRP]], [[vHost]]
-
* Backstore devices: [[IBLOCK]], [[FILEIO]] and [[RAMDISK]]
+
* [[ConfigFS]] (low level kernel API)
-
* Distributions: [[RTS Director]], RHEL 4.x/5.x, SLES11, [[CentOS]], [[Debian]], [[Ubuntu]], etc.
+
-
* [[ConfigFS]] low level kernel API: [[iSCSI/configFS]] and [[Target/configFS]]
+
-
{{AdSense See also}}
+
== Notes ==
== Notes ==
Line 306: Line 287:
== External links ==
== External links ==
-
* {{Official|http://www.risingtidesystems.com RisingTide Systems}}
+
* {{Official|http://www.daterainc.com}}
-
* [http://www.risingtidesystems.com/doc/rtsadmin-gpl/html/rtsadmin_reference.html RTSadmin] Reference Guide
+
* {{LIO Admin Manual}}
-
* [http://www.risingtidesystems.com/doc/rtslib-gpl/html/ RTSlib] Referemce Guide
+
* RTSlib Reference Guide {{Lib Ref Guide HTML}}{{Lib Ref Guide PDF}}
-
* [http://git.kernel.org/?p=linux/kernel/git/nab/lio-core-2.6.git;a=tree;f=drivers/target/tcm_loop;hb=HEAD <code>tcm_loop.ko</code>]
+
 
 +
{{LIO Timeline}}
[[Category:Fabric modules]]
[[Category:Fabric modules]]
[[Category:SCSI]]
[[Category:SCSI]]
[[Category:Virtualization]]
[[Category:Virtualization]]

Latest revision as of 02:36, 7 August 2015

LinuxIO
Logo
LIO 150513.png
tcm_loop SCSI Emulation Device
Original author(s) Nicholas Bellinger
Developer(s) Datera, Inc.
Initial release May 18, 2011 (2011-05-18)
Stable release 4.1.0 / June 20, 2012;
8 years ago
 (2012-06-20)
Preview release 4.2.0-rc5 / June 28, 2012;
8 years ago
 (2012-06-28)
Development status Production
Written in C
Operating system Linux
Type Fabric module
License GNU General Public License, version 2 (GPLv2)
Website datera.io
See LIO for a complete overview over all fabric modules.
FILEIO with a tcm_loop SAS port on KVM x86_64 host running v2.6.34 with the new LSI Megasas 8708EM2 PPC HBA emulation into a Windows 7 x64 KVM Guest.

tcm_loop is a fabric module that can emulate any supported fabric type locally for LinuxIO.

Contents

Overview

tcm_loop provides full local SCSI device emulation (SPC-3 and SPC-4) for raw block storage objects. It thus makes any raw block device apear to the LinuxIO as a full SPC-3/SPC-4 SCSI compliant device with the corresponding CDB-level SAS emulation, for access from any SCSI LLD driver.

tcm_loop minimizes overhead - it delivers >12 GB/s (native) and >2 GB/s (from a single KVM guest, without hardware assist) on Intel Nehalem platforms. This provides e.g., unmodified guest operating systems, such as VMware ESX or VMware vSphere, to use the full SPC-3/SPC-4 SCSI functionality on any type of local raw block storage object they require.

The tcm_loop fabric module (loopback.ko, Linux kernel driver database) for LIO was released with Linux kernel 2.6.39 on May 18, 2011 (2011-05-18).[1]

targetcli

targetcli from Datera, Inc. is used to configure tcm_loop targets. targetcli aggregates LIO service modules via a core library, and exports them through an API, to provide a unified single-node SAN configuration shell, independently of the underlying fabric(s).

Startup

targetcli is invoked by running targetcli as root from the command prompt of the underlying LIO shell. An initial setup as described in targetcli is assumed.

# targetcli
Welcome to targetcli:

 Copyright (c) 2014 by Datera, Inc.
 All rights reserved.

Visit us at http://www.datera.io.

Using ib_srpt fabric module.
Using qla2xxx fabric module.
Using iscsi fabric module.
Using loopback fabric module.

/> loopback/ info
Fabric module name: loopback
ConfigFS path: /sys/kernel/config/target/loopback
Allowed WWN list type: naa
Fabric module specfile: /var/target/fabric/loopback.spec
Fabric module features: nexus
Corresponding kernel module: tcm_loop
/>

Cheat sheet

Command Comment
/backstores/iblock create my_disk /dev/sdb Create the LUN my_disk on the device /dev/sdb
In /backstores/iblock/my_disk>:
/loopback create
Create a local loopback target
/saveconfig Commit the configuration

Display the object tree

Use ls to list the object hierarchy, which is initially empty:

/> ls
o- / ..................................................................... [...]
  o- backstores .......................................................... [...]
  | o- fileio ............................................... [0 Storage Object]
  | o- iblock ............................................... [0 Storage Object]
  | o- pscsi ................................................ [0 Storage Object]
  | o- rd_dr ................................................ [0 Storage Object]
  | o- rd_mcp ............................................... [0 Storage Object]
  o- ib_srpt ........................................................ [0 Target]
  o- iscsi .......................................................... [0 Target]
  o- loopback ....................................................... [0 Target]
  o- qla2xxx ........................................................ [0 Target]
/>

Per default, auto_cd_after_create is set to true, which automatically enters an object context (or working directory) after its creation. The examples here are modeled after this behavior.

Optionally, set auto_cd_after_create=false to prevent targetcli from automatically entering new object contexts after their creation:

/> set global auto_cd_after_create=false
Parameter auto_cd_after_create is now 'false'.
/>

Create a backstore

Create a backstore using the IBLOCK or FILEIO type devices.

For instance, enter the top-level backstore context and create an IBLOCK backstore from a /dev/sdb block device:

/> cd backstores/
/backstores> iblock/ create name=my_disk dev=/dev/sdb
Generating a wwn serial.
Created iblock storage object my_disk using /dev/sdb.
Entering new node /backstores/iblock/my_disk.
/backstores/iblock/my_disk>

targetcli automatically creates a WWN serial ID for the backstore device and then changes the working context to it.

The resulting object hierarchy looks as follows (displayed from the root object):

/> ls
o- / ..................................................................... [...]
  o- backstores .......................................................... [...]
  | o- fileio ............................................... [0 Storage Object]
  | o- iblock ............................................... [1 Storage Object]
  | | o- my_disk .......................................... [/dev/sdb activated]
  | o- pscsi ................................................ [0 Storage Object]
  | o- rd_dr ................................................ [0 Storage Object]
  | o- rd_mcp ............................................... [0 Storage Object]
  o- ib_srpt ........................................................ [0 Target]
  o- iscsi .......................................................... [0 Target]
  o- loopback ....................................................... [0 Target]
  o- qla2xxx ........................................................ [0 Target]
/>

Alternatively, any LVM logical volume can be used as a backstore, please refer to the LIO Admin Manual on how to create them properly.

For instance, create an IBLOCK backstore on a logical volume (under /dev/<volume_group_name>/<logical_volume_name>):

/backstores> iblock/ create name=my_disk dev=/dev/vg0/lv1
Generating a wwn serial.
Created iblock storage object my_disk using /dev/vg0/lv1.
Entering new node /backstores/iblock/my_disk.
/backstores/iblock/my_disk>

Again, targetcli automatically creates a WWN serial ID for the backstore device and then changes the working context to it.

Instantiate a target

Instantiate a loopback target on the existing IBLOCK backstore device my_disk (as set up in targetcli):

/backstores/iblock/my_disk> /loopback create
Successfully created LUN 0.
Entering new node /loopback/naa.60014057f822d991/luns/lun0.
/loopback/naa...991/luns/lun0> 

targetcli automatically assigns the default ID '0' to the LUN, and then changes the working context to the new LUN. Now the target is created, and exports /dev/sdb as LUN 0.

Return to the root context, as no attributes need to be set or modified for a standard loopback target:

/loopback/naa...991/luns/lun0> cd /
/>

Display the object tree

The resulting object hierarchy looks as follows (displayed from the root object):

/> ls
o- / ..................................................................... [...]
  o- backstores .......................................................... [...]
  | o- fileio ............................................... [0 Storage Object]
  | o- iblock ............................................... [1 Storage Object]
  | | o- my_disk .......................................... [/dev/sdb activated]
  | o- pscsi ................................................ [0 Storage Object]
  | o- rd_dr ................................................ [0 Storage Object]
  | o- rd_mcp ............................................... [0 Storage Object]
  o- ib_srpt ........................................................ [0 Target]
  o- iscsi .......................................................... [0 Target]
  o- loopback ....................................................... [1 Target]
  | o- naa.60014057f822d991 ............................. [naa.60014053485b9490]
  | | o- luns .......................................................... [1 LUN]
  | |   o- lun0 .................................... [iblock/my_disk (/dev/sbd)]
  o- qla2xxx ........................................................ [0 Target]
/>

Persist the configuration

Use saveconfig from the root context to persist the LIO configuration across OS reboots:

/> saveconfig
WARNING: Saving rtsnode1 current configuration to disk will overwrite your boot settings.
The current target configuration will become the default boot config.
Are you sure? Type 'yes': yes
Making backup of srpt/ConfigFS with timestamp: 2012-02-27_23:19:37.660264
Successfully updated default config /etc/target/srpt_start.sh
Making backup of qla2xxx/ConfigFS with timestamp: 2012-02-27_23:19:37.660264
Successfully updated default config /etc/target/qla2xxx_start.sh
Making backup of loopback/ConfigFS with timestamp: 2012-02-27_23:19:37.660264
Successfully updated default config /etc/target/loopback_start.sh
Making backup of LIO-Target/ConfigFS with timestamp: 2012-02-27_23:19:37.660264
Successfully updated default config /etc/target/lio_backup-2012-02-27_23:19:37.660264.sh
Making backup of Target_Core_Mod/ConfigFS with timestamp: 2012-02-27_23:19:37.660264
Successfully updated default config /etc/target/tcm_backup-2012-02-27_23:19:37.660264.sh
Generated Target_Core_Mod config: /etc/target/backup/tcm_backup-2012-02-27_23:19:37.660264.sh
Successfully updated default config /etc/target/lio_start.sh
Successfully updated default config /etc/target/tcm_start.sh
/>

Spec file

Datera spec files define the fabric-dependent feature set, capabilities and available target ports of the specific underlying fabric. In particular, the tcm_loop spec file /var/target/fabric/loopback.spec is included via RTSlib.

# WARNING: This is a draft specfile supplied for demo purposes only.

# The loopback fabric module uses the default feature set
features = nexus

# Use naa WWNs
wwn_type = naa

# Non-standard module naming scheme
kernel_module = tcm_loop

# The configfs group
configfs_group = loopback

Specifications

The type of SCSI fabric features that are available with tcm_loop (target mode emulated SAS ports) includes the complete set of SPC-3/SPC-4 logic in the LIO engine, including advanced features such as PRs, ALUA, VAAI, etc.

tcm_loop supports high level fabric emulation. That is, depending on which WWN is used to create the ConfigFS group in /sys/kernel/config/target/loopback/$WWN, it configures itself to return iSCSI, FC etc. device identifiers for the SCSI LUN protocol identifiers device identifier page.

See also

Notes

  1. Linus Torvalds (2011-05-18). "Linux 2.6.39". lkml.org. 

External links

Timeline of the LinuxIO
Release Details 2011 2012 2013 2014 2015
123456789101112 123456789101112 123456789101112 123456789101112 123456789101112
4.x Version 4.0 4.1
Feature LIO Core Loop back FCoE iSCSI Perf SRP
CM WQ FC
USB
1394
vHost Perf Misc 16 GFC iSER Misc VAAI Misc DIF Core
NPIV
DIF iSER DIF FC vhost TCMU Xen Misc Misc virtio 1.0 Misc NVMe OF
Linux 2.6.38 2.6.39 3.0 3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9 3.10 3.11 3.12 3.13 3.14 3.15 3.16 3.17 3.18 3.19 3.20 3.21 3.22
Personal tools
Namespaces
Variants
Actions
Navigation
Toolbox
Google AdSense