tcm_loop
The Linux SCSI Target Wiki
m |
m |
||
(15 intermediate revisions not shown) | |||
Line 1: | Line 1: | ||
{{DISPLAYTITLE:tcm_loop}} | {{DISPLAYTITLE:tcm_loop}} | ||
{{Infobox software | {{Infobox software | ||
- | | name = | + | | name = {{Target}} |
| logo = [[Image:Corp_Logo.png|180px|Logo]] | | logo = [[Image:Corp_Logo.png|180px|Logo]] | ||
| screenshot = {{RTS screenshot|tcm_loop}} | | screenshot = {{RTS screenshot|tcm_loop}} | ||
Line 22: | Line 22: | ||
| language = | | language = | ||
| genre = Fabric module | | genre = Fabric module | ||
- | | license = | + | | license = {{GPLv2}} |
| website = {{RTS website}} | | website = {{RTS website}} | ||
}} | }} | ||
- | :''See [[ | + | :''See [[LIO]] for a complete overview over all fabric modules.'' |
{{Image|TCM-KVM-Megasas-8708EM2-Windows7-x64.png|[[FILEIO]] with a tcm_loop SAS port on [[Kernel-based Virtual Machine|KVM]] x86_64 host running v2.6.34 with the new LSI Megasas 8708EM2 PPC HBA emulation into a Windows 7 x64 [[Kernel-based Virtual Machine|KVM]] Guest.}} | {{Image|TCM-KVM-Megasas-8708EM2-Windows7-x64.png|[[FILEIO]] with a tcm_loop SAS port on [[Kernel-based Virtual Machine|KVM]] x86_64 host running v2.6.34 with the new LSI Megasas 8708EM2 PPC HBA emulation into a Windows 7 x64 [[Kernel-based Virtual Machine|KVM]] Guest.}} | ||
<!-- | <!-- | ||
{{Image|TCM Loop-OS2Warp-QEMU-KVM-HPFS.png|[[FILEIO]] with a tcm_loop SCSI port on [[Kernel-based Virtual Machine|KVM]] x86_64 host running v2.6.36-rc3+ with [[QEMU]] lsi53c895a HBA emulation into an OS/2 Warp v4 guest. A HPFS partition has been formatted and mounted as DRIVE D:.}} | {{Image|TCM Loop-OS2Warp-QEMU-KVM-HPFS.png|[[FILEIO]] with a tcm_loop SCSI port on [[Kernel-based Virtual Machine|KVM]] x86_64 host running v2.6.36-rc3+ with [[QEMU]] lsi53c895a HBA emulation into an OS/2 Warp v4 guest. A HPFS partition has been formatted and mounted as DRIVE D:.}} | ||
- | {{Image|TCM_Loop-3.2-RHEL5-NetBSD5.png|tcm_loop fabric module running on | + | {{Image|TCM_Loop-3.2-RHEL5-NetBSD5.png|tcm_loop fabric module running on {{T}} providing shared virtual SAS target emulation into a VM hypervisor to multiple VM guests of RHEL 5.4 x86_64 and NetBSD 5.x 32-bit.}} |
- | {{Image|TCM Loop-3.2-RHEL5-vmw6.png|tcm_loop fabric module running on | + | {{Image|TCM Loop-3.2-RHEL5-vmw6.png|tcm_loop fabric module running on {{T}} v3.2 on Linux Host v2.6.32-rc5 providing persistent reservations to RHEL 5.4 x86_64 [[Kernel-based Virtual Machine|KVM]] Guest using scsi generic passthrough with VMware Workstation 6.5.}} |
--> | --> | ||
- | '''tcm_loop''' is a fabric module that can emulate any supported fabric type locally for | + | '''tcm_loop''' is a fabric module that can emulate any supported fabric type locally for {{Target}}. |
== Overview == | == Overview == | ||
- | tcm_loop provides full local SCSI device emulation ([[SPC-3]] and [[SPC-4]]) for raw block storage objects. It thus makes any raw block device apear to the | + | tcm_loop provides full local SCSI device emulation ([[SPC-3]] and [[SPC-4]]) for raw block storage objects. It thus makes any raw block device apear to the {{Target}} as a full [[SPC-3]]/[[SPC-4]] [[SCSI]] compliant device with the corresponding [[CDB]]-level SAS emulation, for access from any SCSI LLD driver. |
tcm_loop minimizes overhead - it delivers >12 GB/s (native) and >2 GB/s (from a single [[Kernel-based Virtual Machine|KVM]] guest, without [http://en.wikipedia.org/wiki/Hardware-assisted_virtualization hardware assist]) on Intel [http://en.wikipedia.org/wiki/Nehalem_(microarchitecture) Nehalem] platforms. This provides e.g., unmodified guest operating systems, such as [[VMware ESX]] or [[VMware vSphere]], to use the full [[SPC-3]]/[[SPC-4]] [[SCSI]] functionality on any type of local raw block storage object they require. | tcm_loop minimizes overhead - it delivers >12 GB/s (native) and >2 GB/s (from a single [[Kernel-based Virtual Machine|KVM]] guest, without [http://en.wikipedia.org/wiki/Hardware-assisted_virtualization hardware assist]) on Intel [http://en.wikipedia.org/wiki/Nehalem_(microarchitecture) Nehalem] platforms. This provides e.g., unmodified guest operating systems, such as [[VMware ESX]] or [[VMware vSphere]], to use the full [[SPC-3]]/[[SPC-4]] [[SCSI]] functionality on any type of local raw block storage object they require. | ||
- | The tcm_loop fabric module ({{RTS releases|tcm_loop|module_repo}}, {{RTS releases|tcm_loop|module_info}}) for | + | The tcm_loop fabric module ({{RTS releases|tcm_loop|module_repo}}, {{RTS releases|tcm_loop|module_info}}) for {{T}} was released with Linux kernel {{RTS releases|tcm_loop|kernel_ver}} on {{RTS releases|tcm_loop|initial_date}}.<ref>{{RTS releases|tcm_loop|kernel_rel}}</ref> |
== targetcli == | == targetcli == | ||
- | ''[[targetcli]]'' from {{RTS full}} is used to configure tcm_loop targets. ''targetcli'' aggregates service modules via a core library, and exports them through an API | + | ''[[targetcli]]'' from {{RTS full}} is used to configure tcm_loop targets. ''targetcli'' aggregates {{T}} service modules via a core library, and exports them through an API, to provide a unified single-node SAN configuration shell, independently of the underlying fabric(s). |
- | {{Ambox| type=info| head= | + | {{Ambox| type=info| head=[[LIO]] Admin Manual| text=The {{LIO Admin Manual}} provides comprehensive background and many examples on using ''targetcli'' and on programming the {{RTS short}} library.}} |
=== Startup === | === Startup === | ||
- | [[targetcli]] is invoked by running ''targetcli'' as root from the command prompt of the underlying OS shell. An initial setup as described in targetcli is assumed. | + | [[targetcli]] is invoked by running ''targetcli'' as root from the command prompt of the underlying {{OS}} shell. An initial setup as described in targetcli is assumed. |
<pre> | <pre> | ||
Line 58: | Line 58: | ||
Welcome to targetcli: | Welcome to targetcli: | ||
- | Copyright (c) | + | Copyright (c) 2014 by Datera, Inc. |
All rights reserved. | All rights reserved. | ||
- | Visit us at http://www. | + | Visit us at http://www.datera.io. |
Using ib_srpt fabric module. | Using ib_srpt fabric module. | ||
Line 78: | Line 78: | ||
</pre> | </pre> | ||
- | === | + | === Cheat sheet === |
- | + | ||
- | + | ||
{| class="table-left" | {| class="table-left" | ||
Line 166: | Line 164: | ||
</pre> | </pre> | ||
- | Alternatively, any LVM logical volume can be used as a backstore, please refer to the | + | Alternatively, any LVM logical volume can be used as a backstore, please refer to the {{LIO Admin Manual}} on how to create them properly. |
For instance, create an IBLOCK backstore on a logical volume (under ''/dev/<volume_group_name>/<logical_volume_name>''): | For instance, create an IBLOCK backstore on a logical volume (under ''/dev/<volume_group_name>/<logical_volume_name>''): | ||
Line 228: | Line 226: | ||
=== Persist the configuration === | === Persist the configuration === | ||
- | {{Ambox| type=warning| head=Don't forget to use ''saveconfig''!| text=Without ''saveconfig'', the | + | {{Ambox| type=warning| head=Don't forget to use ''saveconfig''!| text=Without ''saveconfig'', the {{T}} configuration will be lost upon rebooting or unloading the target service, as it will revert back to the last saved one.}} |
- | Use ''saveconfig'' from the root context to persist the | + | Use ''saveconfig'' from the root context to persist the {{T}} configuration across OS reboots: |
<pre> | <pre> | ||
Line 255: | Line 253: | ||
=== Spec file === | === Spec file === | ||
- | RTS spec files define the fabric-dependent feature set, capabilities and available target ports of the specific underlying fabric. In particular, the tcm_loop spec file ''/var/target/fabric/loopback.spec'' is included via RTSlib. | + | {{RTS short}} spec files define the fabric-dependent feature set, capabilities and available target ports of the specific underlying fabric. In particular, the tcm_loop spec file ''/var/target/fabric/loopback.spec'' is included via RTSlib. |
<pre> | <pre> | ||
Line 275: | Line 273: | ||
== Specifications == | == Specifications == | ||
- | The type of SCSI fabric features that are available with tcm_loop ( | + | The type of [[SCSI]] fabric features that are available with tcm_loop (target mode emulated SAS ports) includes the complete set of SPC-3/SPC-4 logic in the {{T}} engine, including advanced features such as [[PRs]], [[ALUA]], [[VAAI]], etc. |
tcm_loop supports high level fabric emulation. That is, depending on which [[WWN]] is used to create the [[ConfigFS]] group in ''/sys/kernel/config/target/loopback/$WWN'', it configures itself to return iSCSI, FC etc. device identifiers for the SCSI LUN protocol identifiers device identifier page. | tcm_loop supports high level fabric emulation. That is, depending on which [[WWN]] is used to create the [[ConfigFS]] group in ''/sys/kernel/config/target/loopback/$WWN'', it configures itself to return iSCSI, FC etc. device identifiers for the SCSI LUN protocol identifiers device identifier page. | ||
== See also == | == See also == | ||
- | * [[ | + | * [[{{OS}}]] |
- | * | + | * {{Target}}, [[targetcli]] |
- | + | * [[FCoE]], [[Fibre Channel]], [[iSCSI]], [[iSER]], [[SRP]], [[vHost]] | |
* [[ConfigFS]] (low level kernel API) | * [[ConfigFS]] (low level kernel API) | ||
Line 289: | Line 287: | ||
== External links == | == External links == | ||
- | * {{Official|http://www.daterainc.com | + | * {{Official|http://www.daterainc.com}} |
- | * | + | * {{LIO Admin Manual}} |
- | * RTSlib Reference Guide | + | * RTSlib Reference Guide {{Lib Ref Guide HTML}}{{Lib Ref Guide PDF}} |
{{LIO Timeline}} | {{LIO Timeline}} |
Latest revision as of 02:36, 7 August 2015
![]() | |
---|---|
![]() tcm_loop SCSI Emulation Device | |
Original author(s) | Nicholas Bellinger |
Developer(s) | Datera, Inc. |
Initial release | May 18, 2011 |
Stable release | 4.1.0 / June 20, 2012 |
Preview release | 4.2.0-rc5 / June 28, 2012 |
Development status | Production |
Written in | C |
Operating system | Linux |
Type | Fabric module |
License | GNU General Public License, version 2 (GPLv2) |
Website | datera.io |
- See LIO for a complete overview over all fabric modules.
tcm_loop is a fabric module that can emulate any supported fabric type locally for LinuxIO.
Contents |
Overview
tcm_loop provides full local SCSI device emulation (SPC-3 and SPC-4) for raw block storage objects. It thus makes any raw block device apear to the LinuxIO as a full SPC-3/SPC-4 SCSI compliant device with the corresponding CDB-level SAS emulation, for access from any SCSI LLD driver.
tcm_loop minimizes overhead - it delivers >12 GB/s (native) and >2 GB/s (from a single KVM guest, without hardware assist) on Intel Nehalem platforms. This provides e.g., unmodified guest operating systems, such as VMware ESX or VMware vSphere, to use the full SPC-3/SPC-4 SCSI functionality on any type of local raw block storage object they require.
The tcm_loop fabric module (loopback.ko, Linux kernel driver database) for LIO was released with Linux kernel 2.6.39 on May 18, 2011 .[1]
targetcli
targetcli from Datera, Inc. is used to configure tcm_loop targets. targetcli aggregates LIO service modules via a core library, and exports them through an API, to provide a unified single-node SAN configuration shell, independently of the underlying fabric(s).
![]() |
LIO Admin Manual The LIO Admin Manual provides comprehensive background and many examples on using targetcli and on programming the Datera library. |
Startup
targetcli is invoked by running targetcli as root from the command prompt of the underlying LIO shell. An initial setup as described in targetcli is assumed.
# targetcli Welcome to targetcli: Copyright (c) 2014 by Datera, Inc. All rights reserved. Visit us at http://www.datera.io. Using ib_srpt fabric module. Using qla2xxx fabric module. Using iscsi fabric module. Using loopback fabric module. /> loopback/ info Fabric module name: loopback ConfigFS path: /sys/kernel/config/target/loopback Allowed WWN list type: naa Fabric module specfile: /var/target/fabric/loopback.spec Fabric module features: nexus Corresponding kernel module: tcm_loop />
Cheat sheet
Command | Comment |
---|---|
/backstores/iblock create my_disk /dev/sdb | Create the LUN my_disk on the device /dev/sdb |
In /backstores/iblock/my_disk>: /loopback create | Create a local loopback target |
/saveconfig | Commit the configuration |
Display the object tree
Use ls to list the object hierarchy, which is initially empty:
/> ls o- / ..................................................................... [...] o- backstores .......................................................... [...] | o- fileio ............................................... [0 Storage Object] | o- iblock ............................................... [0 Storage Object] | o- pscsi ................................................ [0 Storage Object] | o- rd_dr ................................................ [0 Storage Object] | o- rd_mcp ............................................... [0 Storage Object] o- ib_srpt ........................................................ [0 Target] o- iscsi .......................................................... [0 Target] o- loopback ....................................................... [0 Target] o- qla2xxx ........................................................ [0 Target] />
![]() |
Global parameter auto_cd_after_create After the creation of a new object, automatically enter its object context. |
Per default, auto_cd_after_create is set to true, which automatically enters an object context (or working directory) after its creation. The examples here are modeled after this behavior.
Optionally, set auto_cd_after_create=false to prevent targetcli from automatically entering new object contexts after their creation:
/> set global auto_cd_after_create=false Parameter auto_cd_after_create is now 'false'. />
Create a backstore
Create a backstore using the IBLOCK or FILEIO type devices.
For instance, enter the top-level backstore context and create an IBLOCK backstore from a /dev/sdb block device:
/> cd backstores/ /backstores> iblock/ create name=my_disk dev=/dev/sdb Generating a wwn serial. Created iblock storage object my_disk using /dev/sdb. Entering new node /backstores/iblock/my_disk. /backstores/iblock/my_disk>
targetcli automatically creates a WWN serial ID for the backstore device and then changes the working context to it.
The resulting object hierarchy looks as follows (displayed from the root object):
/> ls o- / ..................................................................... [...] o- backstores .......................................................... [...] | o- fileio ............................................... [0 Storage Object] | o- iblock ............................................... [1 Storage Object] | | o- my_disk .......................................... [/dev/sdb activated] | o- pscsi ................................................ [0 Storage Object] | o- rd_dr ................................................ [0 Storage Object] | o- rd_mcp ............................................... [0 Storage Object] o- ib_srpt ........................................................ [0 Target] o- iscsi .......................................................... [0 Target] o- loopback ....................................................... [0 Target] o- qla2xxx ........................................................ [0 Target] />
Alternatively, any LVM logical volume can be used as a backstore, please refer to the LIO Admin Manual on how to create them properly.
For instance, create an IBLOCK backstore on a logical volume (under /dev/<volume_group_name>/<logical_volume_name>):
/backstores> iblock/ create name=my_disk dev=/dev/vg0/lv1 Generating a wwn serial. Created iblock storage object my_disk using /dev/vg0/lv1. Entering new node /backstores/iblock/my_disk. /backstores/iblock/my_disk>
Again, targetcli automatically creates a WWN serial ID for the backstore device and then changes the working context to it.
![]() |
More backstore examples More examples on creating backstores can be found in targetcli. |
Instantiate a target
Instantiate a loopback target on the existing IBLOCK backstore device my_disk (as set up in targetcli):
/backstores/iblock/my_disk> /loopback create Successfully created LUN 0. Entering new node /loopback/naa.60014057f822d991/luns/lun0. /loopback/naa...991/luns/lun0>
targetcli automatically assigns the default ID '0' to the LUN, and then changes the working context to the new LUN. Now the target is created, and exports /dev/sdb as LUN 0.
Return to the root context, as no attributes need to be set or modified for a standard loopback target:
/loopback/naa...991/luns/lun0> cd / />
Display the object tree
The resulting object hierarchy looks as follows (displayed from the root object):
/> ls o- / ..................................................................... [...] o- backstores .......................................................... [...] | o- fileio ............................................... [0 Storage Object] | o- iblock ............................................... [1 Storage Object] | | o- my_disk .......................................... [/dev/sdb activated] | o- pscsi ................................................ [0 Storage Object] | o- rd_dr ................................................ [0 Storage Object] | o- rd_mcp ............................................... [0 Storage Object] o- ib_srpt ........................................................ [0 Target] o- iscsi .......................................................... [0 Target] o- loopback ....................................................... [1 Target] | o- naa.60014057f822d991 ............................. [naa.60014053485b9490] | | o- luns .......................................................... [1 LUN] | | o- lun0 .................................... [iblock/my_disk (/dev/sbd)] o- qla2xxx ........................................................ [0 Target] />
Persist the configuration
![]() |
Don't forget to use saveconfig! Without saveconfig, the LIO configuration will be lost upon rebooting or unloading the target service, as it will revert back to the last saved one. |
Use saveconfig from the root context to persist the LIO configuration across OS reboots:
/> saveconfig WARNING: Saving rtsnode1 current configuration to disk will overwrite your boot settings. The current target configuration will become the default boot config. Are you sure? Type 'yes': yes Making backup of srpt/ConfigFS with timestamp: 2012-02-27_23:19:37.660264 Successfully updated default config /etc/target/srpt_start.sh Making backup of qla2xxx/ConfigFS with timestamp: 2012-02-27_23:19:37.660264 Successfully updated default config /etc/target/qla2xxx_start.sh Making backup of loopback/ConfigFS with timestamp: 2012-02-27_23:19:37.660264 Successfully updated default config /etc/target/loopback_start.sh Making backup of LIO-Target/ConfigFS with timestamp: 2012-02-27_23:19:37.660264 Successfully updated default config /etc/target/lio_backup-2012-02-27_23:19:37.660264.sh Making backup of Target_Core_Mod/ConfigFS with timestamp: 2012-02-27_23:19:37.660264 Successfully updated default config /etc/target/tcm_backup-2012-02-27_23:19:37.660264.sh Generated Target_Core_Mod config: /etc/target/backup/tcm_backup-2012-02-27_23:19:37.660264.sh Successfully updated default config /etc/target/lio_start.sh Successfully updated default config /etc/target/tcm_start.sh />
Spec file
Datera spec files define the fabric-dependent feature set, capabilities and available target ports of the specific underlying fabric. In particular, the tcm_loop spec file /var/target/fabric/loopback.spec is included via RTSlib.
# WARNING: This is a draft specfile supplied for demo purposes only. # The loopback fabric module uses the default feature set features = nexus # Use naa WWNs wwn_type = naa # Non-standard module naming scheme kernel_module = tcm_loop # The configfs group configfs_group = loopback
Specifications
The type of SCSI fabric features that are available with tcm_loop (target mode emulated SAS ports) includes the complete set of SPC-3/SPC-4 logic in the LIO engine, including advanced features such as PRs, ALUA, VAAI, etc.
tcm_loop supports high level fabric emulation. That is, depending on which WWN is used to create the ConfigFS group in /sys/kernel/config/target/loopback/$WWN, it configures itself to return iSCSI, FC etc. device identifiers for the SCSI LUN protocol identifiers device identifier page.
See also
Notes
- ↑ Linus Torvalds (2011-05-18). "Linux 2.6.39". lkml.org.
External links
- Official website
- LIO Admin Manual
- RTSlib Reference Guide [HTML][PDF]
Timeline of the LinuxIO | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
Release | Details | 2011 | 2012 | 2013 | 2014 | 2015 | |||||||||||||||||||||||||||||||||||||||||||||||||||||||
1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | ||
4.x | Version | 4.0 | 4.1 | ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Feature | LIO Core | Loop back | FCoE | iSCSI | Perf | SRP | CM WQ | FC USB 1394 | vHost | Perf | Misc | 16 GFC | iSER | Misc | VAAI | Misc | DIF Core NPIV | DIF iSER | DIF FC vhost | TCMU Xen | Misc | Misc | virtio 1.0 | Misc | NVMe OF | ||||||||||||||||||||||||||||||||||||
Linux | 2.6.38 | 2.6.39 | 3.0 | 3.1 | 3.2 | 3.3 | 3.4 | 3.5 | 3.6 | 3.7 | 3.8 | 3.9 | 3.10 | 3.11 | 3.12 | 3.13 | 3.14 | 3.15 | 3.16 | 3.17 | 3.18 | 3.19 | 3.20 | 3.21 | 3.22 |