Fibre Channel

The Linux SCSI Target Wiki

(Difference between revisions)
Jump to: navigation, search
m
m
 
(86 intermediate revisions not shown)
Line 1: Line 1:
{{Infobox software
{{Infobox software
-
| name                  = LIO Target
+
| name                  = {{Target}}
-
| logo                  = [[Image:RisingTide_Logo_small.png|180px|Logo]]
+
| logo                  = [[Image:Corp_Logo.png|180px|Logo]]
| screenshot            = {{RTS screenshot|FC-QLogic}}
| screenshot            = {{RTS screenshot|FC-QLogic}}
| caption                = Fibre Channel fabric module(s)
| caption                = Fibre Channel fabric module(s)
Line 24: Line 24:
| website                = {{RTS website}}
| website                = {{RTS website}}
}}
}}
-
:''See [[Target]] for a complete overview over all fabric modules.''
+
:''See [[LIO]] for a complete overview over all fabric modules.''
{{Image|Tcm qla2xxx-pcie-passthrough-to-baremetal.png|QLogic Fibre Channel running at line rate in target mode with PCIe device passthrough and MSI-X polled interrupts across Linux/SCSI qla2xxx LLD request and response rings.}}
{{Image|Tcm qla2xxx-pcie-passthrough-to-baremetal.png|QLogic Fibre Channel running at line rate in target mode with PCIe device passthrough and MSI-X polled interrupts across Linux/SCSI qla2xxx LLD request and response rings.}}
Line 35: Line 35:
Fibre Channel is standardized in the [http://www.t11.org/index.html T11] Technical Committee of the Inter National Committee for Information Technology Standards (INCITS), an American National Standards Institute (ANSI) - accredited standards committee.
Fibre Channel is standardized in the [http://www.t11.org/index.html T11] Technical Committee of the Inter National Committee for Information Technology Standards (INCITS), an American National Standards Institute (ANSI) - accredited standards committee.
-
Fibre Channel was used primarily in the supercomputer field, but has become the standard connection type for storage area networks (SAN) in enterprise storage. Despite its name, Fibre Channel signaling can run on both twisted pair copper wire and fiber-optic cables.
+
Fibre Channel has been the standard connection type for storage area networks (SAN) in enterprise storage. Despite its name, Fibre Channel signaling can run on both twisted pair copper wire and fiber-optic cables.
-
The Fibre Channel Protocol (FCP) is a transport protocol (similar to TCP used in IP networks) which predominantly transports SCSI commands over Fibre Channel networks.
+
The Fibre Channel Protocol (FCP) is a transport protocol which predominantly transports SCSI commands over Fibre Channel networks.
== Hardware support ==
== Hardware support ==
-
The following Fiber Channel HBAs and ASICs from [http://www.qlogic.com QLogic] are supported:
+
The following [http://www.qlogic.com QLogic] Fiber Channel [http://www.qlogic.com/Products/adapters/Pages/FibreChannelAdapters.aspx HBAs] are supported in 4/8-gigabit mode:
-
* QLogic 2400 Series (single-port QLx246x), 4GBFC
+
* QLogic 2400 Series (QLx246x), 4GFC
-
* QLogic 2500 Series (dual-port QLE256x), 8GBFC
+
* QLogic 2500 Series (QLE256x), 8GFC (fully qual'd)
-
* QLogic 2600 Series (dual-port), 16 GBFC, incl. high-speed and SR-IOV extensions
+
-
The QLogic fabric module ({{RTS releases|FC-QLogic|module_repo}}) went upstream into the Linux kernel {{RTS releases|FC-QLogic|kernel_ver}} on 5/31/2012.<ref>{{RTS releases|FC-QLogic|kernel_up}}</ref>
+
The QLogic Fibre Channel fabric module ({{RTS releases|FC-QLogic|module_repo}}, {{RTS releases|FC-QLogic|module_info}}) for the Linux [[SCSI]] [[Target]] was released with Linux kernel {{RTS releases|FC-QLogic|kernel_ver}} on {{RTS releases|FC-QLogic|initial_date}}.<ref>{{RTS releases|FC-QLogic|kernel_rel}}</ref>
 +
 
 +
With Linux 3.9, the following 16-gigabit QLogic Fibre Channel HBA is supported, which makes {{T}} the first open source target to support 16GFC:
 +
 
 +
* QLogic 2600 Series (QLE266x), 16GFC, SR-IOV
 +
 
 +
With Linux 3.9, the following QLogic [http://www.qlogic.com/Products/adapters/Pages/ConvergedNetworkAdapters.aspx CNAs] are also supported:
 +
 
 +
* QLogic 8300 Series (QLE834x), 16GFS/10 GbE, PCIe Gen3 SR-IOV
 +
* QLogic 8100 Series (QLE81xx), 8GFC/10 GbE, PCIe Gen2
== Enable target mode ==
== Enable target mode ==
-
By default, the upstream qla2xxx driver runs in initiator mode. To use it with LIO, first enable Fibre Channel target mode with the corresponding ''qlini_mode'' module parameter.
+
By default, the upstream qla2xxx driver runs in initiator mode. To use it with {{T}}, first enable Fibre Channel target mode with the corresponding ''qlini_mode'' module parameter.<ref>{{cite web |url=http://www.spinics.net/lists/target-devel/msg03108.html |title=Re: targetcli qla2xxx create fails |author=Nicholas Bellinger |publisher=spinics.net |date=2012-09-05}}</ref>
-
To enable target mode, add the following parameter to the qla2xxx module configuration file. Depending on your distribution, the specific module configuration file might be different, e.g., for [[Debian]] it is ''/etc/modprobe.d/qla2xxx.conf'':<ref>{{cite web |url=http://www.spinics.net/lists/target-devel/msg03108.html |title=Re: targetcli qla2xxx create fails |author=Nicholas Bellinger| publisher=spinics.net| date=9/5/2012}}</ref>
+
To enable target mode, add the following parameter to the qla2xxx module configuration file:
<pre>
<pre>
options qla2xxx qlini_mode="disabled"
options qla2xxx qlini_mode="disabled"
</pre>
</pre>
 +
 +
Depending on your distribution, the module configuration file might be different, for instance:
 +
 +
* /etc/modprobe.d/qla2xxx.conf: [[CentOS]], [[Debian]], [[Fedora]], [[RHEL]], [[Scientific Linux]]
 +
* /etc/modprobe.conf.local: [[openSUSE]], SLES
 +
 +
In order for these changes to take effect, the ''initrd/initramfs'' will need to be rebuilt.
Please verify that ''initrd/initramfs'' is accepting the additional qla2xxx parameter.
Please verify that ''initrd/initramfs'' is accepting the additional qla2xxx parameter.
Line 63: Line 78:
== targetcli ==
== targetcli ==
-
''[[targetcli]]'' from {{RTS full}} is used to configure Fibre Channel targets. ''targetcli'' aggregates service modules via a core library, and exports them through an API to the Unified [[Target]], to provide a unified single-node SAN configuration shell, independently of the underlying fabric(s).
+
''[[targetcli]]'' from {{RTS full}} is used to configure Fibre Channel targets. ''targetcli'' aggregates {{T}} service modules via a core library, and exports them through an API, to provide a unified single-node SAN configuration shell, independently of the underlying fabric(s).
 +
 
 +
{{Ambox| type=info| head=[[LIO]] Admin Manual| text={{LIO Admin Manual}} provides comprehensive background and many examples on using ''targetcli'' and on programming the RTS library.}}
 +
 
 +
=== Cheat sheet ===
 +
 
 +
{| class="table-left"
 +
|- bgcolor=#a0c0f0
 +
! Command
 +
! Comment
 +
|-
 +
| ''/backstores/iblock create my_disk /dev/sdb''
 +
| Create the LUN ''my_disk'' on the block device ''/dev/sdb''
 +
|-
 +
| ''/qla2xxx create <WWPN>''
 +
| Create a Fibre Channel target
 +
|-
 +
| In /qla2xxx/<WWPN>:<br/>''luns/ create /backstores/iblock/my_disk''
 +
| Export the LUN ''my_disk''
 +
|-
 +
| In /qla2xxx/<WWPN>:<br/>''acls/ create <Initiator WWPN>''
 +
| Allow access for the initiator at ''<WWPN>''
 +
|-
 +
| ''/saveconfig''
 +
| Commit the configuration
 +
|-
 +
|}
=== Startup ===
=== Startup ===
-
[[targetcli]] is invoked by running ''targetcli'' as root from the command prompt of the underlying OS shell.
+
[[targetcli]] is invoked by running ''targetcli'' as root from the command prompt of the underlying [[{{OS}}]] shell.
<pre>
<pre>
Line 73: Line 114:
Welcome to targetcli:
Welcome to targetcli:
-
  Copyright (c) 2012 by RisingTide Systems LLC.
+
  Copyright (c) 2014 by Datera, Inc.
  All rights reserved.
  All rights reserved.
-
Visit us at http://www.risingtidesystems.com.
+
Visit us at http://www.datera.io.
Using ib_srpt fabric module.
Using ib_srpt fabric module.
Line 115: Line 156:
</pre>
</pre>
-
{{Message/tip|The global parameter ''auto_cd_after_create''.|Automatically enter new object context after their creation.}}
+
{{Ambox| type=info| head=Global parameter ''auto_cd_after_create''| text=After the creation of a new object, automatically enter its object context.}}
-
Per default, ''auto_cd_after_create'' is set to ''true'', which automatically enters the object context (or working directory) of new objects after their creation. Set ''auto_cd_after_create=false'' to prevent RTSadmin from automatically entering the object context to new objects after their creation:
+
Per default, ''auto_cd_after_create'' is set to ''true'', which automatically enters an object context (or working directory) after its creation. The examples here are modeled after this behavior.
 +
 
 +
Optionally, set ''auto_cd_after_create=false'' to prevent ''targetcli'' from automatically entering new object contexts after their creation:
<pre>
<pre>
Line 127: Line 170:
=== Create a backstore ===
=== Create a backstore ===
-
Enter the top-level backstore object, and create one (storage object) using IBLOCK or FILEIO type devices.
+
Create a backstore using the IBLOCK or FILEIO type devices.
-
For instance, create an IBLOCK backstore from a ''/dev/sdb'' block device. Note that this device can be any TYPE_DISK block-device, and it can also use ''/dev/disk/by-id/'' symlinks:
+
For instance, enter the top-level ''backstore'' context and create an IBLOCK backstore from a ''/dev/sdb'' block device:
<pre>
<pre>
Line 160: Line 203:
/>
/>
</pre>
</pre>
 +
 +
Alternatively, any LVM logical volume can be used as a backstore, please refer to the {{LIO Admin Manual}} on how to create them properly.
 +
 +
For instance, create an IBLOCK backstore on a logical volume (under ''/dev/<volume_group_name>/<logical_volume_name>''):
 +
 +
<pre>
 +
/backstores> iblock/ create name=my_disk dev=/dev/vg0/lv1
 +
Generating a wwn serial.
 +
Created iblock storage object my_disk using /dev/vg0/lv1.
 +
Entering new node /backstores/iblock/my_disk.
 +
/backstores/iblock/my_disk>
 +
</pre>
 +
 +
Again, ''targetcli'' automatically creates a WWN serial ID for the backstore devices and then changes the working context to it.
 +
 +
{{Ambox| type=info| head=More backstore examples| text=More examples on creating backstores can be found in ''[[targetcli#Create a backstore|targetcli]]''.}}
=== Instantiate a target ===
=== Instantiate a target ===
Line 168: Line 227:
* 21:00:00:24:ff:31:4c:49
* 21:00:00:24:ff:31:4c:49
-
Instantiate a Fibre Channel target, in this example for QLogic HBAs, on the existing IBLOCK backstore device ''my_disk'':
+
Instantiate a Fibre Channel target, in this example for QLogic HBAs, on the existing IBLOCK backstore device ''my_disk'' (as set up in [[targetcli]]):
<pre>
<pre>
Line 203: Line 262:
=== Define access rights ===
=== Define access rights ===
-
Configure the access rights to allow logins from initiators. This requires setting up individual access rights for each initiators, based on its WWPN.
+
Configure the access rights to allow logins from initiators. This requires setting up individual access rights for each initiator, based on its WWPN.
-
Determine the WWPN for the respective Fibre Channel initiator. For Linux initiator systems, e.g., use:
+
Determine the WWPN for the respective Fibre Channel initiator. For instance, for Linux initiator systems, use:
<pre>
<pre>
Line 211: Line 270:
</pre>
</pre>
-
For a simple setup, allow access to the initiator with the WWPN as determined above:
+
For a simple setup, grant access to the initiator with the WWPN as determined above:
<pre>
<pre>
Line 222: Line 281:
</pre>
</pre>
-
''targetcli'' then automatically adds the appropriate mapped LUNs per default.
+
''targetcli'' per default automatically adds the appropriate mapped LUNs.
=== Display the object tree ===
=== Display the object tree ===
Line 253: Line 312:
=== Persist the configuration ===
=== Persist the configuration ===
-
The target configuration can be persisted across OS reboots by using ''saveconfig'' from the root context:
+
{{Ambox| type=warning| head=Don't forget to use ''saveconfig''!| text=Without ''saveconfig'', the {{T}} configuration will be lost upon rebooting or unloading the target service, as the target configuration will revert back to the last saved one.}}
 +
 
 +
Use ''saveconfig'' from the root context to persist the target configuration across OS reboots:
<pre>
<pre>
Line 275: Line 336:
/>
/>
</pre>
</pre>
-
 
-
{{Message/warning|Don't forget to use ''saveconfig''!|Without ''saveconfig'', the target configuration is ephemeral and will be lost upon rebooting or unloading the target service.}}
 
=== Spec file ===
=== Spec file ===
-
RTS spec files define the fabric-dependent feature set, capabilities and available target ports of the specific underlying fabric.
+
{{RTS short}} spec files define the fabric-dependent feature set, capabilities and available target ports of the specific underlying fabric.
In particular, the QLogic spec file ''/var/target/fabric/qla2xxx.spec'' is included via RTSlib. WWN values are extracted via ''/sys/class/fc_host/host*/port_name'' in ''wwn_from_files_filter'', and are presented in the [[targetcli]] [[WWN]] working context to register individual Fibre Channel port GUIDs.
In particular, the QLogic spec file ''/var/target/fabric/qla2xxx.spec'' is included via RTSlib. WWN values are extracted via ''/sys/class/fc_host/host*/port_name'' in ''wwn_from_files_filter'', and are presented in the [[targetcli]] [[WWN]] working context to register individual Fibre Channel port GUIDs.
Line 301: Line 360:
# The configfs group is default
# The configfs group is default
# configfs_group = qla2xxx
# configfs_group = qla2xxx
 +
</pre>
 +
 +
== Scripting with RTSlib ==
 +
 +
=== Setup script ===
 +
 +
The following Python code illustrates how to setup a basic Fibre Channel target and export a mapped
 +
LUN:
 +
 +
<pre>
 +
#!/usr/bin/python
 +
# Fibre Channel setup script example with RTSlib
 +
from rtslib import *
 +
 +
# Setup an IBLOCK backstore
 +
backstore = IBlockBackstore(3, mode='create')
 +
try:
 +
    so = IBlockStorageObject(backstore, "fioa", "/dev/fioa", gen_wwn=True)
 +
except:
 +
    backstore.delete()
 +
    raise
 +
 +
# Create an FC target endpoint using a qla2xxx WWPN
 +
fabric = FabricModule('qla2xxx')
 +
target = Target(fabric, '21:00:00:24:ff:31:4c:48')
 +
tpg = TPG(target, 1)
 +
 +
# Export LUN 0 via the 'so' StorageObject class
 +
lun0 = tpg.lun(0, so, "my_lun")
 +
 +
# Setup the NodeACL for an FC initiator, and create MappedLUN 0
 +
node_acl = tpg.node_acl('21:00:00:24:ff:31:4c:4c')
 +
mapped_lun = node_acl.mapped_lun(0, 0, False)
 +
</pre>
 +
 +
Note that while Fibre Channel TPGs are masked by [[targetcli]], they are not masked by RTSlib.
 +
 +
=== Object tree ===
 +
 +
The resulting object tree looks as follows:
 +
 +
<pre>
 +
o- / ..................................................................... [...]
 +
  o- backstores .......................................................... [...]
 +
  | o- fileio ............................................... [0 Storage Object]
 +
  | o- iblock ............................................... [1 Storage Object]
 +
  | | o- my_disk .......................................... [/dev/sdb activated]
 +
  | o- pscsi ................................................ [0 Storage Object]
 +
  | o- rd_dr ................................................ [0 Storage Object]
 +
  | o- rd_mcp ............................................... [0 Storage Object]
 +
  o- ib_srpt ........................................................ [0 Target]
 +
  o- iscsi .......................................................... [0 Target]
 +
  o- loopback ....................................................... [0 Target]
 +
  o- qla2xxx ........................................................ [1 Target]
 +
    o- 21:00:00:24:ff:31:4c:48 ....................................... [enabled]
 +
      o- acls .......................................................... [0 ACL]
 +
      | o- 21:00:00:24:ff:31:4c:4c .............................. [1 Mapped LUN]
 +
      |  o- mapped_lun0 ........................................... [lun0 (rw)]
 +
      o- luns .......................................................... [1 LUN]
 +
        o- lun0 .................................... [iblock/my_disk (/dev/sdb)]
</pre>
</pre>
Line 307: Line 426:
{{anchor|T10}}The following specifications are available as [http://www.t10.org/drafts.htm T10 Working Drafts]:
{{anchor|T10}}The following specifications are available as [http://www.t10.org/drafts.htm T10 Working Drafts]:
-
* {{anchor|FCP}}'''Fibre Channel Protocol''' ('''FCP'''): FCP defines the protocol to be used to transport SCSI commands over the T11 Fibre Channel interface, 12/4/1995
+
* {{anchor|FCP}}'''Fibre Channel Protocol''' ('''FCP'''): FCP defines the protocol to be used to transport SCSI commands over the T11 Fibre Channel interface, 1995-12-04
-
* {{anchor|FCP-2}}'''SCSI Fibre Channel Protocol - 2''' ('''FCP-2'''): FCP-2 defines the second generation Fibre Channel Protocol to be used to transport SCSI commands over the T11 Fibre Channel interface, 10/23/2002
+
* {{anchor|FCP-2}}'''SCSI Fibre Channel Protocol - 2''' ('''FCP-2'''): FCP-2 defines the second generation Fibre Channel Protocol to be used to transport SCSI commands over the T11 Fibre Channel interface, 2002-10-23
-
* {{anchor|FCP-3}}'''Fibre Channel Protocol - 3''' ('''FCP-3'''): FCP-3 defines the third generation Fibre Channel Protocol to be used to transport SCSI commands over the T11 Fibre Channel interface, 9/13/2005
+
* {{anchor|FCP-3}}'''Fibre Channel Protocol - 3''' ('''FCP-3'''): FCP-3 defines the third generation Fibre Channel Protocol to be used to transport SCSI commands over the T11 Fibre Channel interface, 2005-09-13
-
* {{anchor|FCP-3}}'''Fibre Channel Protocol - 4''' ('''FCP-4'''): FCP-4 defines the fouth generation Fibre Channel Protocol to be used to transport SCSI commands over the T11 Fibre Channel interface, 11/9/2010
+
* {{anchor|FCP-3}}'''Fibre Channel Protocol - 4''' ('''FCP-4'''): FCP-4 defines the fouth generation Fibre Channel Protocol to be used to transport SCSI commands over the T11 Fibre Channel interface, 2010-11-09
== Glossary ==
== Glossary ==
Line 324: Line 443:
* {{RFC|4626|MIB for Fibre Channel's Fabric Shortest Path First (FSPF) Protocol}}
* {{RFC|4626|MIB for Fibre Channel's Fabric Shortest Path First (FSPF) Protocol}}
{{Clear}}
{{Clear}}
-
 
-
== Timeline ==
 
-
{{LIO Timeline}}
 
== See also ==
== See also ==
-
* [[RTS OS]], [[targetcli]]
+
* [[{{OS}}]]
-
* [[Target]]
+
* {{Target}}, [[targetcli]]
-
* [[Fibre Channel over Ethernet|FCoE]], [[InfiniBand]], [[iSCSI]], [[tcm_loop]], [[vHost]]
+
* [[FCoE]], [[iSCSI]], [[iSER]], [[SRP]], [[tcm_loop]], [[vHost]]
== Notes ==
== Notes ==
Line 337: Line 453:
== External links ==
== External links ==
-
* [[RTS OS]] [http://www.risingtidesystems.com/doc/RTS%20OS%20Admin%20Manual%20CE.pdf Admin Manual]
+
* {{LIO Admin Manual}}
-
* RTSlib Reference Guide [[http://www.risingtidesystems.com/doc/rtslib-gpl/html/ HTML]][[http://www.risingtidesystems.com/doc/rtslib-gpl/pdf/rtslib-API-reference.pdf PDF]]
+
* RTSlib Reference Guide {{Lib Ref Guide HTML}}{{Lib Ref Guide PDF}}
* [http://en.wikipedia.org/wiki/Fibre_Channel Fibre Channel] Wikipedia entry
* [http://en.wikipedia.org/wiki/Fibre_Channel Fibre Channel] Wikipedia entry
* [http://en.wikipedia.org/wiki/QLogic QLogic] Wikipedia entry
* [http://en.wikipedia.org/wiki/QLogic QLogic] Wikipedia entry
Line 344: Line 460:
* [http://www.emulex.com/ Emulex] website
* [http://www.emulex.com/ Emulex] website
* [http://www.t11.org/index.html T11] home page
* [http://www.t11.org/index.html T11] home page
 +
* [http://filedownloads.qlogic.com/files/driver/48437/README_qla2xxx2-6U4.htm QLogic README]
 +
 +
{{LIO Timeline}}
[[Category:Fabric modules]]
[[Category:Fabric modules]]
[[Category:Fibre Channel]]
[[Category:Fibre Channel]]
[[Category:Network protocols]]
[[Category:Network protocols]]
-
__NOEDITSECTION__
 

Latest revision as of 02:38, 7 August 2015

LinuxIO
Logo
LIO 150513.png
Fibre Channel fabric module(s)
Original author(s) Nicholas Bellinger
Andrew Vasquez
Madhu Iyengar
Developer(s) Datera, Inc.
Initial release July 21, 2012 (2012-07-21)
Stable release 4.1.0 / June 20, 2012;
8 years ago (2012-06-20)
Preview release 4.2.0-rc5 / June 28, 2012;
8 years ago (2012-06-28)
Development status Production
Written in C
Operating system Linux
Type Fabric module
License GNU General Public License
Website datera.io
See LIO for a complete overview over all fabric modules.
QLogic Fibre Channel running at line rate in target mode with PCIe device passthrough and MSI-X polled interrupts across Linux/SCSI qla2xxx LLD request and response rings.
TCM v4.0 qla2xxx using PCIe device passthrough for target mode MSI-X interrupts, and QEMU Megasas HBA emulation into Windows7 64-bit guest.

Fibre Channel (FC) provides drivers for various FC Host Bus Adapters (HBAs). Fibre Channel is a gigabit-speed network technology primarily used for storage networking.

Contents

Overview

Fibre Channel is standardized in the T11 Technical Committee of the Inter National Committee for Information Technology Standards (INCITS), an American National Standards Institute (ANSI) - accredited standards committee.

Fibre Channel has been the standard connection type for storage area networks (SAN) in enterprise storage. Despite its name, Fibre Channel signaling can run on both twisted pair copper wire and fiber-optic cables.

The Fibre Channel Protocol (FCP) is a transport protocol which predominantly transports SCSI commands over Fibre Channel networks.

Hardware support

The following QLogic Fiber Channel HBAs are supported in 4/8-gigabit mode:

The QLogic Fibre Channel fabric module (qla2xxx.ko, Linux kernel driver database) for the Linux SCSI Target was released with Linux kernel 3.5 on July 21, 2012 (2012-07-21).[1]

With Linux 3.9, the following 16-gigabit QLogic Fibre Channel HBA is supported, which makes LIO the first open source target to support 16GFC:

With Linux 3.9, the following QLogic CNAs are also supported:

Enable target mode

By default, the upstream qla2xxx driver runs in initiator mode. To use it with LIO, first enable Fibre Channel target mode with the corresponding qlini_mode module parameter.[2]

To enable target mode, add the following parameter to the qla2xxx module configuration file: 

options qla2xxx qlini_mode="disabled"

Depending on your distribution, the module configuration file might be different, for instance:

In order for these changes to take effect, the initrd/initramfs will need to be rebuilt.

Please verify that initrd/initramfs is accepting the additional qla2xxx parameter.

targetcli

targetcli from Datera, Inc. is used to configure Fibre Channel targets. targetcli aggregates LIO service modules via a core library, and exports them through an API, to provide a unified single-node SAN configuration shell, independently of the underlying fabric(s).

LIO Admin Manual
LIO Admin Manual provides comprehensive background and many examples on using targetcli and on programming the RTS library.

Cheat sheet

Command Comment
/backstores/iblock create my_disk /dev/sdb Create the LUN my_disk on the block device /dev/sdb
/qla2xxx create <WWPN> Create a Fibre Channel target
In /qla2xxx/<WWPN>:
luns/ create /backstores/iblock/my_disk 		Export the LUN my_disk
In /qla2xxx/<WWPN>:
acls/ create <Initiator WWPN> 		Allow access for the initiator at <WWPN>
/saveconfig Commit the configuration

Startup

targetcli is invoked by running targetcli as root from the command prompt of the underlying LIO shell. 

# targetcli
Welcome to targetcli:

 Copyright (c) 2014 by Datera, Inc.
 All rights reserved.

Visit us at http://www.datera.io.

Using ib_srpt fabric module.
Using qla2xxx fabric module.
Using iscsi fabric module.
Using loopback fabric module.

/> qla2xxx/ info
Fabric module name: qla2xxx
ConfigFS path: /sys/kernel/config/target/qla2xxx
Allowed WWN list type: free
Fabric module specfile: /var/target/fabric/qla2xxx.spec
Fabric module features: acls
Corresponding kernel module: tcm_qla2xxx
/>

Upon targetcli initialization, the underlying RTSlib loads the installed fabric modules, and creates the corresponding ConfigFS mount points (at /sys/kernel/config/target/<fabric>), as specified by the associated spec files (located in /var/target/fabric/fabric.spec).

Display the object tree

Use ls to list the object hierarchy, which is initially empty: 

/> ls
o- / ..................................................................... [...]
  o- backstores .......................................................... [...]
  | o- fileio ............................................... [0 Storage Object]
  | o- iblock ............................................... [0 Storage Object]
  | o- pscsi ................................................ [0 Storage Object]
  | o- rd_dr ................................................ [0 Storage Object]
  | o- rd_mcp ............................................... [0 Storage Object]
  o- ib_srpt ........................................................ [0 Target]
  o- iscsi .......................................................... [0 Target]
  o- loopback ....................................................... [0 Target]
  o- qla2xxx ........................................................ [0 Target]
/>
Global parameter auto_cd_after_create
After the creation of a new object, automatically enter its object context.

Per default, auto_cd_after_create is set to true, which automatically enters an object context (or working directory) after its creation. The examples here are modeled after this behavior.

Optionally, set auto_cd_after_create=false to prevent targetcli from automatically entering new object contexts after their creation: 

/> set global auto_cd_after_create=false
Parameter auto_cd_after_create is now 'false'.
/>

Create a backstore

Create a backstore using the IBLOCK or FILEIO type devices.

For instance, enter the top-level backstore context and create an IBLOCK backstore from a /dev/sdb block device: 

/> cd backstores/
/backstores> iblock/ create name=my_disk dev=/dev/sdb
Generating a wwn serial.
Created iblock storage object my_disk using /dev/sdb.
Entering new node /backstores/iblock/my_disk.
/backstores/iblock/my_disk>

targetcli automatically creates a WWN serial ID for the backstore device and then changes the working context to it.

The resulting object hierarchy looks as follows (displayed from the root object): 

/> ls
o- / ..................................................................... [...]
  o- backstores .......................................................... [...]
  | o- fileio ............................................... [0 Storage Object]
  | o- iblock ............................................... [1 Storage Object]
  | | o- my_disk .......................................... [/dev/sdb activated]
  | o- pscsi ................................................ [0 Storage Object]
  | o- rd_dr ................................................ [0 Storage Object]
  | o- rd_mcp ............................................... [0 Storage Object]
  o- ib_srpt ........................................................ [0 Target]
  o- iscsi .......................................................... [0 Target]
  o- loopback ....................................................... [0 Target]
  o- qla2xxx ........................................................ [0 Target]
/>

Alternatively, any LVM logical volume can be used as a backstore, please refer to the LIO Admin Manual on how to create them properly.

For instance, create an IBLOCK backstore on a logical volume (under /dev/<volume_group_name>/<logical_volume_name>): 

/backstores> iblock/ create name=my_disk dev=/dev/vg0/lv1
Generating a wwn serial.
Created iblock storage object my_disk using /dev/vg0/lv1.
Entering new node /backstores/iblock/my_disk.
/backstores/iblock/my_disk>

Again, targetcli automatically creates a WWN serial ID for the backstore devices and then changes the working context to it.

More backstore examples
More examples on creating backstores can be found in targetcli.

Instantiate a target

The Fibre Channel ports that are available on the storage array are presented in the WWN context with the following WWNPs, for instance:

Instantiate a Fibre Channel target, in this example for QLogic HBAs, on the existing IBLOCK backstore device my_disk (as set up in targetcli): 

/backstores/iblock/my_disk> /qla2xxx create 21:00:00:24:ff:31:4c:48
Created target 21:00:00:24:ff:31:4c:48.
Entering new node /qla2xxx/21:00:00:24:ff:31:4c:48.
/qla2xxx/21:0...4:ff:31:4c:48>

targetcli automatically changes the working context to the resulting tagged Endpoint.

Export LUNs

Declare LUNs for the backstore device, to form a valid SAN storage object: 

/qla2xxx/21:0...4:ff:31:4c:48> luns/ create /backstores/iblock/my_disk
Selected LUN 0.
Successfully created LUN 0.
Entering new node /qla2xxx/21:00:00:24:ff:31:4c:48/luns/lun0.
/qla2xxx/21:0...:48/luns/lun0>

targetcli per default automatically assigns the default ID '0' to the LUN, and then changes the working context to the new SAN storage object. The target is now created, and exports /dev/sdb as LUN 0.

Return to the underlying Endpoint as the working context, as no attributes need to be set or modified for standard LUNs: 

/qla2xxx/21:0...:48/luns/lun0> cd <
Taking you back to /qla2xxx/21:00:00:24:ff:31:4c:48.
/qla2xxx/21:0...4:ff:31:4c:48>

Define access rights

Configure the access rights to allow logins from initiators. This requires setting up individual access rights for each initiator, based on its WWPN.

Determine the WWPN for the respective Fibre Channel initiator. For instance, for Linux initiator systems, use: 

# cat /sys/class/fc_host/host*/port_name | sed -e s/0x// -e 's/../&:/g' -e s/:$//

For a simple setup, grant access to the initiator with the WWPN as determined above: 

/qla2xxx/21:0...4:ff:31:4c:48> acls/ create 21:00:00:24:ff:31:4c:4c
Successfully created Node ACL for 21:00:00:24:ff:31:4c:4c.
Created mapped LUN 0.
Entering new node /qla2xxx/21:00:00:24:ff:31:4c:48/acls/21:00:00:24:ff:31:4c:4c.
/qla2xxx/21:0...4:ff:31:4c:4c> cd /
/>

targetcli per default automatically adds the appropriate mapped LUNs.

Display the object tree

The resulting Fibre Channel SAN object hierarchy looks as follows (displayed from the root object): 

/> ls
o- / ..................................................................... [...]
  o- backstores .......................................................... [...]
  | o- fileio ............................................... [0 Storage Object]
  | o- iblock ............................................... [1 Storage Object]
  | | o- my_disk .......................................... [/dev/sdb activated]
  | o- pscsi ................................................ [0 Storage Object]
  | o- rd_dr ................................................ [0 Storage Object]
  | o- rd_mcp ............................................... [0 Storage Object]
  o- ib_srpt ........................................................ [0 Target]
  o- iscsi .......................................................... [0 Target]
  o- loopback ....................................................... [0 Target]
  o- qla2xxx ........................................................ [1 Target]
    o- 21:00:00:24:ff:31:4c:48 ....................................... [enabled]
      o- acls .......................................................... [1 ACL]
      | o- 21:00:00:24:ff:31:4c:4c .............................. [1 Mapped LUN]
      |   o- mapped_lun0 ........................................... [lun0 (rw)]
      o- luns .......................................................... [1 LUN]
        o- lun0 .................................... [iblock/my_disk (/dev/sdb)]
/>

Persist the configuration

Don't forget to use saveconfig!
Without saveconfig, the LIO configuration will be lost upon rebooting or unloading the target service, as the target configuration will revert back to the last saved one.

Use saveconfig from the root context to persist the target configuration across OS reboots: 

/> saveconfig
WARNING: Saving rtsnode1 current configuration to disk will overwrite your boot settings.
The current target configuration will become the default boot config.
Are you sure? Type 'yes': yes
Making backup of srpt/ConfigFS with timestamp: 2012-02-27_23:19:37.660264
Successfully updated default config /etc/target/srpt_start.sh
Making backup of qla2xxx/ConfigFS with timestamp: 2012-02-27_23:19:37.660264
Successfully updated default config /etc/target/qla2xxx_start.sh
Making backup of loopback/ConfigFS with timestamp: 2012-02-27_23:19:37.660264
Successfully updated default config /etc/target/loopback_start.sh
Making backup of LIO-Target/ConfigFS with timestamp: 2012-02-27_23:19:37.660264
Successfully updated default config /etc/target/lio_backup-2012-02-27_23:19:37.660264.sh
Making backup of Target_Core_Mod/ConfigFS with timestamp: 2012-02-27_23:19:37.660264
Successfully updated default config /etc/target/tcm_backup-2012-02-27_23:19:37.660264.sh
Generated Target_Core_Mod config: /etc/target/backup/tcm_backup-2012-02-27_23:19:37.660264.sh
Successfully updated default config /etc/target/lio_start.sh
Successfully updated default config /etc/target/tcm_start.sh
/>

Spec file

Datera spec files define the fabric-dependent feature set, capabilities and available target ports of the specific underlying fabric.

In particular, the QLogic spec file /var/target/fabric/qla2xxx.spec is included via RTSlib. WWN values are extracted via /sys/class/fc_host/host*/port_name in wwn_from_files_filter, and are presented in the targetcli WWN working context to register individual Fibre Channel port GUIDs. 

# WARNING: This is a draft specfile supplied for demo purposes only.

# The qla2xxx fabric module uses the default feature set.
features = acls

# Non-standard module naming scheme
kernel_module = tcm_qla2xxx

# The module uses hardware addresses from there
wwn_from_files = /sys/class/fc_host/host*/port_name

# Transform '0x1234567812345678' WWN notation to '12:34:56:78:12:34:56:78'
wwn_from_files_filter = "sed -e s/0x// -e 's/../&:/g' -e s/:$//"

# The configfs group is default
# configfs_group = qla2xxx

Scripting with RTSlib

Setup script

The following Python code illustrates how to setup a basic Fibre Channel target and export a mapped LUN: 

#!/usr/bin/python
# Fibre Channel setup script example with RTSlib
from rtslib import *

# Setup an IBLOCK backstore
backstore = IBlockBackstore(3, mode='create')
try:
    so = IBlockStorageObject(backstore, "fioa", "/dev/fioa", gen_wwn=True)
except:
    backstore.delete()
    raise

# Create an FC target endpoint using a qla2xxx WWPN
fabric = FabricModule('qla2xxx')
target = Target(fabric, '21:00:00:24:ff:31:4c:48')
tpg = TPG(target, 1)

# Export LUN 0 via the 'so' StorageObject class
lun0 = tpg.lun(0, so, "my_lun")

# Setup the NodeACL for an FC initiator, and create MappedLUN 0
node_acl = tpg.node_acl('21:00:00:24:ff:31:4c:4c')
mapped_lun = node_acl.mapped_lun(0, 0, False)

Note that while Fibre Channel TPGs are masked by targetcli, they are not masked by RTSlib.

Object tree

The resulting object tree looks as follows: 

o- / ..................................................................... [...]
  o- backstores .......................................................... [...]
  | o- fileio ............................................... [0 Storage Object]
  | o- iblock ............................................... [1 Storage Object]
  | | o- my_disk .......................................... [/dev/sdb activated]
  | o- pscsi ................................................ [0 Storage Object]
  | o- rd_dr ................................................ [0 Storage Object]
  | o- rd_mcp ............................................... [0 Storage Object]
  o- ib_srpt ........................................................ [0 Target]
  o- iscsi .......................................................... [0 Target]
  o- loopback ....................................................... [0 Target]
  o- qla2xxx ........................................................ [1 Target]
    o- 21:00:00:24:ff:31:4c:48 ....................................... [enabled]
      o- acls .......................................................... [0 ACL]
      | o- 21:00:00:24:ff:31:4c:4c .............................. [1 Mapped LUN]
      |   o- mapped_lun0 ........................................... [lun0 (rw)]
      o- luns .......................................................... [1 LUN]
        o- lun0 .................................... [iblock/my_disk (/dev/sdb)]

Specifications

The following specifications are available as T10 Working Drafts:

Glossary

RFCs

See also

Notes

  1. ↑ Linus Torvalds (2012-07-21). "Linux 3.5 released". marc.info. 
  2. ↑ Nicholas Bellinger (2012-09-05). "Re: targetcli qla2xxx create fails". spinics.net. 

External links

Timeline of the LinuxIO
Release Details 2011 2012 2013 2014 2015
123456789101112 123456789101112 123456789101112 123456789101112 123456789101112
4.x Version 4.0 4.1
Feature LIO Core Loop back FCoE iSCSI Perf SRP
 		CM WQ FC
USB
1394 		vHost Perf Misc 16 GFC iSER Misc VAAI Misc DIF Core
NPIV 		DIF iSER DIF FC vhost TCMU Xen Misc Misc virtio 1.0 Misc NVMe OF
Linux 2.6.38 2.6.39 3.0 3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9 3.10 3.11 3.12 3.13 3.14 3.15 3.16 3.17 3.18 3.19 3.20 3.21 3.22
Retrieved from "http://wwww.linux-iscsi.org/wiki/Fibre_Channel"
Personal tools
Namespaces
Variants
Actions
Navigation
Toolbox
Google AdSense