ISCSI Extensions for RDMA

The Linux SCSI Target Wiki

(Difference between revisions)
Jump to: navigation, search
m (Overview)
m
 
(31 intermediate revisions not shown)
Line 1: Line 1:
{{Infobox software
{{Infobox software
-
| name                  = LIO Target
+
| name                  = {{Target}}
-
| logo                  = [[Image:RisingTide_Logo_small.png|180px|Logo]]
+
| logo                  = [[Image:Corp_Logo.png|180px|Logo]]
| screenshot            = {{RTS screenshot|iSER}}
| screenshot            = {{RTS screenshot|iSER}}
| caption                = [[InfiniBand]] iSER fabric module
| caption                = [[InfiniBand]] iSER fabric module
| collapsible            =  
| collapsible            =  
-
| author                = {{Nicholas Bellinger}}
+
| author                = {{Nicholas Bellinger}}<br/>{{Or Gerlitz}}
-
| developer              = {{RTS legal}}
+
| developer              = {{RTS legal}}<br/>Mellanox Technologies, Ltd.
-
| released              = {{RTS releases|iSER-Mellanox|initial_date}}
+
| released              = {{RTS releases|iSER|initial_date}}
-
| discontinued          = {{RTS releases|iSER-Mellanox|eol_date}}
+
| discontinued          = {{RTS releases|iSER|eol_date}}
-
| latest release version = {{RTS releases|iSER-Mellanox|release_ver}}
+
| latest release version = {{RTS releases|iSER|release_ver}}
-
| latest release date    = {{RTS releases|iSER-Mellanox|release_date}}
+
| latest release date    = {{RTS releases|iSER|release_date}}
-
| latest preview version = {{RTS releases|iSER-Mellanox|preview_ver}}
+
| latest preview version = {{RTS releases|iSER|preview_ver}}
-
| latest preview date    = {{RTS releases|iSER-Mellanox|preview_date}}
+
| latest preview date    = {{RTS releases|iSER|preview_date}}
-
| status                = {{RTS releases|iSER-Mellanox|status}}
+
| status                = {{RTS releases|iSER|status}}
| frequently updated    =
| frequently updated    =
| programming language  = C
| programming language  = C
Line 21: Line 21:
| language              =
| language              =
| genre                  = Fabric module
| genre                  = Fabric module
-
| license                = GNU General Public License
+
| license                = {{GPLv2}}
| website                = {{RTS website}}
| website                = {{RTS website}}
}}
}}
-
:''See [[Target]] for a complete overview over all fabric modules.''
+
:''See [[LIO]] for a complete overview over all fabric modules.''
'''iSCSI Extensions for RDMA''' ('''iSER''') is a network protocol that extends [[iSCSI]] to use [[RDMA]].
'''iSCSI Extensions for RDMA''' ('''iSER''') is a network protocol that extends [[iSCSI]] to use [[RDMA]].
Line 30: Line 30:
== Overview ==
== Overview ==
-
iSER permits data to be transferred directly into and out of remote [[SCSI]] computer memory buffers over [[InfiniBand]] and Ethernet networks without intermediate data copies by using RDMA. RDMA is supported on [[InfiniBand]] networks, by [[RoCE]] on "lossless" ([[DCB]]) Ethernet networks, and by [[iWARP]] enhanced [[TOE]] [[NIC]]s over standard Ethernet networks.
+
iSER permits data to be transferred directly into and out of remote [[SCSI]] computer memory buffers over [[InfiniBand]] and Ethernet networks without intermediate data copies by using RDMA. RDMA is supported on [[InfiniBand]] networks, by [[RoCE]] on "lossless" Ethernet ([[DCB]]) networks, and by [[iWARP]] enhanced [[TOE]] [[NIC]]s over standard Ethernet networks.
-
An [[InfiniBand]] iSER fabric module for the Unified [[Target]] is currently being implemented as a joint development effort between {{RTS full}} and [http://mellanox.com Mellanox]. It will be released with Linux kernel version 3.10.
+
The [[InfiniBand]] iSER fabric module for the {{Target}} ({{RTS releases|iSER|module_repo}}, see {{RTS releases|iSER|module_info}}) was implemented as a joint development effort between {{RTS full}} and [http://mellanox.com Mellanox], and released with the Linux kernel {{RTS releases|iSER|kernel_ver}} on {{RTS releases|iSER|initial_date}}.<ref>{{RTS releases|iSER|kernel_rel}}</ref>
== History ==
== History ==
Line 40: Line 40:
== targetcli ==
== targetcli ==
-
''[[targetcli]]'' from {{RTS full}} is used to configure iSER targets. ''targetcli'' aggregates service modules via a core library, and exports them through an API to the Unified [[Target]], to provide a unified single-node SAN configuration shell, independently of the underlying fabric(s).
+
''[[targetcli]]'' from {{RTS full}} is used to configure iSER targets. ''targetcli'' aggregates {{T}} service modules via a core library, and exports them through an API, to provide a unified single-node SAN configuration shell, independently of the underlying fabric(s).
-
{{Ambox| type=info| head=RTS OS Admin Manual| text=The [[RTS OS]] [http://www.risingtidesystems.com/doc/RTS%20OS%20Admin%20Manual%20CE.pdf Admin Manual] provides comprehensive background and many examples on using ''targetcli'' and on programming the RTS library.}}
+
{{Ambox| type=info| head=[[LIO]] Admin Manual| text=The {{LIO Admin Manual}} provides comprehensive background and many examples on using ''targetcli'' and on programming the {{RTS short}} library.}}
-
=== Cookbook ===
+
=== Cheat sheet ===
iSER uses the existing [[iSCSI]] target control plane with a special iSER enable bit per network portal context.
iSER uses the existing [[iSCSI]] target control plane with a special iSER enable bit per network portal context.
-
 
-
The following is a cheat sheet for reference purposes.
 
{| class="table-left"
{| class="table-left"
Line 101: Line 99:
Welcome to targetcli:
Welcome to targetcli:
-
  Copyright (c) 2012 by RisingTide Systems, Inc.
+
  Copyright (c) 2014 by Datera, Inc.
  All rights reserved.
  All rights reserved.
-
Visit us at http://www.risingtidesystems.com.
+
Visit us at http://www.datera.io.
Using ib_srpt fabric module.
Using ib_srpt fabric module.
Line 124: Line 122:
Upon targetcli initialization, the underlying RTSlib loads the installed fabric modules, and creates the corresponding [[ConfigFS]] mount points (at ''/sys/kernel/config/target/<fabric>''), as specified by the associated spec files (located in ''/var/target/fabric/fabric.spec'').
Upon targetcli initialization, the underlying RTSlib loads the installed fabric modules, and creates the corresponding [[ConfigFS]] mount points (at ''/sys/kernel/config/target/<fabric>''), as specified by the associated spec files (located in ''/var/target/fabric/fabric.spec'').
-
=== Display the object hierarchy ===
+
=== Display the object tree ===
Use ''ls'' to list the object hierarchy, which is initially empty:
Use ''ls'' to list the object hierarchy, which is initially empty:
Line 192: Line 190:
</pre>
</pre>
-
Alternatively, any LVM logical volume can be used as a backstore, please refer to the [http://www.risingtidesystems.com/doc/RTS%20OS%20Admin%20Manual%20CE.pdf RTS OS Admin Manual] on how to create them properly.
+
Alternatively, any LVM logical volume can be used as a backstore, please refer to the {{LIO Admin Manual}} on how to create them properly.
For instance, create an IBLOCK backstore on a logical volume (under ''/dev/<volume_group_name>/<logical_volume_name>''):
For instance, create an IBLOCK backstore on a logical volume (under ''/dev/<volume_group_name>/<logical_volume_name>''):
Line 221: Line 219:
</pre>
</pre>
-
''targetcli'' automatically creates the Target Portal Group (TPG) and per default assigns a sequentially increasing TPG tag, starting from '1', thereby creating a TPG1.
+
''targetcli'' automatically creates the Target Portal Group ([[TPG]]) and per default assigns a sequentially increasing TPG tag, starting from '1', thereby creating a TPG1.
=== Export LUNs ===
=== Export LUNs ===
Line 302: Line 300:
* You have carefully analyzed your ACL setup with regard to its security and data integrity requirements and risks.
* You have carefully analyzed your ACL setup with regard to its security and data integrity requirements and risks.
-
==== CHAP Initiator Authentication ====
+
==== CHAP initiator authentication ====
Enable secure sessions for the initiator with the IQN "iqn.1991-05.com.microsoft:ibm-t410s":
Enable secure sessions for the initiator with the IQN "iqn.1991-05.com.microsoft:ibm-t410s":
Line 347: Line 345:
The iSER Endpoint is now ready for secure logins from the specified iSER initiator.
The iSER Endpoint is now ready for secure logins from the specified iSER initiator.
-
==== Mutual CHAP Authentication ====
+
==== Mutual CHAP authentication ====
Enable secure sessions for the initiator with the IQN "iqn.1991-05.com.microsoft:ibm-t410s":
Enable secure sessions for the initiator with the IQN "iqn.1991-05.com.microsoft:ibm-t410s":
Line 396: Line 394:
The iSER Endpoint is now ready for secure logins from the specified iSER initiator.
The iSER Endpoint is now ready for secure logins from the specified iSER initiator.
-
==== TPG Authentication ====
+
==== TPG authentication ====
Setting up authentication information for every single initiator separately can be cumbersome, so ''targetcli'' provides the capability to define common login information for all Endpoints in a TPG. As a result, all initiators connecting to that TPG can use the same login credentials.
Setting up authentication information for every single initiator separately can be cumbersome, so ''targetcli'' provides the capability to define common login information for all Endpoints in a TPG. As a result, all initiators connecting to that TPG can use the same login credentials.
-
Enable TPG Authentication for all Endpoints in a TPG:
+
Enable TPG authentication for all Endpoints in a TPG:
<pre>
<pre>
Line 412: Line 410:
</pre>
</pre>
-
Setup the the common TPG Authentication credentials for all Endpoints in a TPG, which consists of:
+
Setup the the common TPG authentication credentials for all Endpoints in a TPG, which consists of:
* The ''userid'' (login name) and ''password'' (target secret) for the target.
* The ''userid'' (login name) and ''password'' (target secret) for the target.
Line 445: Line 443:
</pre>
</pre>
-
{{Ambox| type=info| head=Individual ACLs| text=Login credentials for specific initiators can be created by adding corresponding ACL entries, as individual ACL entries override common TPG Authentication information.}}
+
{{Ambox| type=info| head=Individual ACLs| text=Login credentials for specific initiators can be created by adding corresponding ACL entries, as individual ACL entries override common TPG authentication information.}}
=== Enable discovery control ===
=== Enable discovery control ===
Line 451: Line 449:
Optionally, the iSER protocol can also control the visibility of iSER targets for discovery by iSER initiators.
Optionally, the iSER protocol can also control the visibility of iSER targets for discovery by iSER initiators.
-
==== CHAP Initiator Discovery Authentication ====
+
==== CHAP initiator discovery authentication ====
Enable CHAP initiator discovery authentication for all initiators by setting up a CHAP ''userid'' (login name) and ''password'' (target secret) in the global ''discovery_auth group'':
Enable CHAP initiator discovery authentication for all initiators by setting up a CHAP ''userid'' (login name) and ''password'' (target secret) in the global ''discovery_auth group'':
Line 465: Line 463:
Only iSER initiators that can authenticate themselves with a user id of "mytargetuid" and a password of "mytargetsecret" can now discover this iSER target.
Only iSER initiators that can authenticate themselves with a user id of "mytargetuid" and a password of "mytargetsecret" can now discover this iSER target.
-
==== Mutual CHAP Discovery Authentication ====
+
==== Mutual CHAP discovery authentication ====
Enable Mutual CHAP discovery authentication for all initiators by setting up Mutual CHAP information in the global ''discovery_auth'' group:
Enable Mutual CHAP discovery authentication for all initiators by setting up Mutual CHAP information in the global ''discovery_auth'' group:
Line 501: Line 499:
Only iSER initiators that can authenticate themselves with a user id of "mytargetuid" and a password of "mytargetsecret" can now discover this iSER target, and conversely, the iSER target can only discover iSER initiators that can authenticate themselves with a user id of "mymutualuid" and a password of "mymutualsecret".
Only iSER initiators that can authenticate themselves with a user id of "mytargetuid" and a password of "mytargetsecret" can now discover this iSER target, and conversely, the iSER target can only discover iSER initiators that can authenticate themselves with a user id of "mymutualuid" and a password of "mymutualsecret".
-
=== Display the object hierarchy ===
+
=== Display the object tree ===
The resulting iSCSI object hierarchy with CHAP Authentication (initiator or mutual) for one initiator looks as follows (displayed from the root object):
The resulting iSCSI object hierarchy with CHAP Authentication (initiator or mutual) for one initiator looks as follows (displayed from the root object):
Line 533: Line 531:
=== Persist the configuration ===
=== Persist the configuration ===
-
{{Ambox| type=warning| head=Don't forget to use ''saveconfig''!| text=Without ''saveconfig'', the target configuration will be lost upon rebooting or unloading the target service, as the target configuration will revert back to the last saved one.}}
+
{{Ambox| type=warning| head=Don't forget to use ''saveconfig''!| text=Without ''saveconfig'', the Linux SCSI Target configuration will be lost upon rebooting or unloading the target service, as the target configuration will revert back to the last saved one.}}
Use ''saveconfig'' from the root context to persist the target configuration across OS reboots:
Use ''saveconfig'' from the root context to persist the target configuration across OS reboots:
Line 560: Line 558:
=== Spec file ===
=== Spec file ===
-
RTS spec files define the fabric-dependent feature set, capabilities and available target ports of the specific underlying fabric.
+
{{RTS short}} spec files define the fabric-dependent feature set, capabilities and available target ports of the specific underlying fabric.
-
In particular, the iSER spec file ''/var/target/fabric/iser.spec'' is included via RTSlib.
+
In particular, iSER uses the [[iSCSI#Spec file|iSCSI spec file]], and does not require a seperate spec file.
 +
 
 +
== Scripting with RTSlib ==
 +
 
 +
=== Setup script ===
 +
 
 +
The following Python code illustrates how to setup a basic iSER target and export a mapped LUN:
<pre>
<pre>
-
# WARNING: This is a draft specfile supplied for demo purposes only.
+
#!/usr/bin/python
 +
# iSER setup script example with RTSlib
 +
from rtslib import *
-
# The iSER fabric module uses the default feature set.
+
# Setup an IBLOCK backstore
-
features = acls, acls_auth, discovery_auth, nps, tpgts
+
backstore = IBlockBackstore(3, mode='create')
 +
try:
 +
    so = IBlockStorageObject(backstore, "sdb", "/dev/sdb", gen_wwn=True)
 +
except:
 +
    backstore.delete()
 +
    raise
-
# Obviously, this module uses IQN strings as WWNs
+
# Create an iSER target endpoint using an iSCSI IQN
-
wwn_type = iqn
+
fabric = FabricModule('iSCSI')
 +
target = Target(fabric, "iqn.2003-01.org.linux-iscsi.x.x8664:sn.d3d8b0500fde")
 +
tpg = TPG(target, 1)
-
# Non-standard module name
+
# Setup a network portal in the iSER TPG
-
kernel_module = ib_isert
+
# The IP address must already be active on the system
 +
portal = NetworkPortal(tpg, "192.168.1.128", "5060")
-
# configfs space maps into the iSCSI tree
+
# Enable iSER for the portal (only difference to iSCSI)
-
configfs_group = iscsi
+
portal._set_iser_attr(1)
 +
 
 +
# Export LUN 0 via the 'so' StorageObject class
 +
lun0 = tpg.lun(0, so, "my_lun")
 +
 
 +
# Setup the NodeACL for an iSER initiator, and create MappedLUN 0
 +
node_acl = tpg.node_acl("iqn.2003-01.org.linux-iscsi.y.x8664:sn.abcdefghijkl")
 +
mapped_lun = node_acl.mapped_lun(0, 0, False)
 +
</pre>
 +
 
 +
=== Object tree ===
 +
 
 +
The resulting object tree looks as follows:
 +
 
 +
<pre>
 +
o- / ..................................................................... [...]
 +
  o- backstores .......................................................... [...]
 +
  | o- fileio ............................................... [0 Storage Object]
 +
  | o- iblock ............................................... [1 Storage Object]
 +
  | | o- sdb .............................................. [/dev/sdb activated]
 +
  | o- pscsi ................................................ [0 Storage Object]
 +
  | o- rd_dr ................................................ [0 Storage Object]
 +
  | o- rd_mcp ............................................... [0 Storage Object]
 +
  o- ib_srpt ........................................................ [0 Target]
 +
  o- iscsi .......................................................... [1 Target]
 +
  | o- iqn.2003-01.org.linux-iscsi.x.x8664:sn.d3d8b0500fde ............. [1 TPG]
 +
  |  o- tpgt1 ....................................................... [enabled]
 +
  |    o- acls ........................................................ [1 ACL]
 +
  |    | o- iqn.2003-01.org.linux-iscsi.y.x8664:sn.abcdefghijkl. [1 Mapped LUN]
 +
  |    |  o- mapped_lun0 ......................................... [lun0 (rw)]
 +
  |    o- luns ........................................................ [1 LUN]
 +
  |    | o- lun0 ...................................... [iblock/sdb (/dev/sdb)]
 +
  |    o- portals .................................................. [1 Portal]
 +
  |      o- 192.168.1.128:5060 ............................. [OK, iser enabled]
 +
  o- loopback ....................................................... [0 Target]
 +
  o- qla2xxx ........................................................ [0 Target]
</pre>
</pre>
Line 592: Line 641:
== See also ==
== See also ==
-
* [[RTS OS]], [[targetcli]]
+
* [[{{OS}}]]
-
* [[Target]]
+
* {{Target}}, [[targetcli]]
-
* [[Fibre Channel]], [[Fibre Channel over Ethernet|FCoE]], [[iSCSI]], [[SRP]], [[tcm_loop]], [[vHost]]
+
* [[FCoE]], [[Fibre Channel]], [[iSCSI]], [[SRP]], [[tcm_loop]], [[vHost]]
* [[ConfigFS]] (low level kernel API)
* [[ConfigFS]] (low level kernel API)
Line 608: Line 657:
== External links ==
== External links ==
-
* [[RTS OS]] [http://www.risingtidesystems.com/doc/RTS%20OS%20Admin%20Manual%20CE.pdf Admin Manual]
+
* {{LIO Admin Manual}}
-
* RTSlib Reference Guide [[http://www.risingtidesystems.com/doc/rtslib-gpl/html/ HTML]][[http://www.risingtidesystems.com/doc/rtslib-gpl/pdf/rtslib-API-reference.pdf PDF]]
+
* RTSlib Reference Guide {{Lib Ref Guide HTML}}{{Lib Ref Guide PDF}}
* [http://en.wikipedia.org/wiki/ISCSI_Extensions_for_RDMA iSER] Wikipedia entry
* [http://en.wikipedia.org/wiki/ISCSI_Extensions_for_RDMA iSER] Wikipedia entry
* [http://marc.info/?l=linux-rdma&r=1&w=2 linux-rdma] mailing list
* [http://marc.info/?l=linux-rdma&r=1&w=2 linux-rdma] mailing list

Latest revision as of 02:39, 7 August 2015

LinuxIO
Logo
LIO 150513.png
InfiniBand iSER fabric module
Original author(s) Nicholas Bellinger
Or Gerlitz
Developer(s) Datera, Inc.
Mellanox Technologies, Ltd.
Initial release June 30, 2013 (2013-06-30)
Stable release 4.1.0 / June 30, 2013;
6 years ago
 (2013-06-30)
Preview release 4.2.0-rc5 / June 30, 2013;
6 years ago
 (2013-06-30)
Development status Production
Written in C
Operating system Linux
Type Fabric module
License GNU General Public License, version 2 (GPLv2)
Website datera.io
See LIO for a complete overview over all fabric modules.

iSCSI Extensions for RDMA (iSER) is a network protocol that extends iSCSI to use RDMA.

Contents

Overview

iSER permits data to be transferred directly into and out of remote SCSI computer memory buffers over InfiniBand and Ethernet networks without intermediate data copies by using RDMA. RDMA is supported on InfiniBand networks, by RoCE on "lossless" Ethernet (DCB) networks, and by iWARP enhanced TOE NICs over standard Ethernet networks.

The InfiniBand iSER fabric module for the LinuxIO (ib_sert.ko, see Linux kernel driver database) was implemented as a joint development effort between Datera, Inc. and Mellanox, and released with the Linux kernel 3.10 on June 30, 2013 (2013-06-30).[1]

History

An RDMA consortium was announced on May 31, 2002 with a goal of product implementations by 2003.[2] The consortium released their proposal in July, 2003.[3] The protocol specifications were published as drafts in September 2004 in the Internet Engineering Task Force and issued as RFCs in October 2007.[4][5] The OpenIB Alliance was renamed in 2007 to be the OpenFabrics Alliance, and then released an open source software package.[6]

targetcli

targetcli from Datera, Inc. is used to configure iSER targets. targetcli aggregates LIO service modules via a core library, and exports them through an API, to provide a unified single-node SAN configuration shell, independently of the underlying fabric(s).

Cheat sheet

iSER uses the existing iSCSI target control plane with a special iSER enable bit per network portal context.

Command Comment
/backstores/iblock create my_disk /dev/sdb Create the LUN my_disk on the device /dev/sdb
/iscsi create Create an iSCSI target
In /iscsi/<IQN>/tpgt1:
portals/ create <IP_address>
Associate an <IP_address>
In /iscsi/<IQN>/tpgt1/<IP_address:port>:
iser_enable
Enable iSER
In /iscsi/<IQN>/tpgt1:
luns/ create /backstores/iblock/my_disk
Export the LUN my_disk
In /iscsi/<IQN>/tpgt1:
set attribute authentication=0
demo_mode_write_protect=0
generate_node_acls=1
cache_dynamic_acls=1
Enable Demo Mode.
Beware!
/saveconfig Commit the configuration

Startup

First, update the rtslib, targetcli and lio-utils packages to the latest revisions.

The iSER target uses RDMA-CM and IPoIB to initiate the iSER login process, so it requires RDMA capable hardware beneath an ibX interface configured with an IP address. The configured ibX interface should look like:

# ifconfig ib0
ib0       Link encap:InfiniBand  HWaddr 80-00-00-48-FE-80-00-00-00-00-00-00-00-00-00-00  
          inet addr:10.100.0.1  Bcast:10.100.0.255  Mask:255.255.255.0
          UP BROADCAST MULTICAST  MTU:4092  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:256 
          RX bytes:0 (0.0 B)  TX bytes:0 (0.0 B)

Then, run targetcli as root from the command prompt of the underlying OS shell.

# targetcli
Welcome to targetcli:

 Copyright (c) 2014 by Datera, Inc.
 All rights reserved.

Visit us at http://www.datera.io.

Using ib_srpt fabric module.
Using qla2xxx fabric module.
Using iscsi fabric module.
Using iser fabric module.
Using loopback fabric module.

/> iser/ info
Fabric module name: iser
ConfigFS path: /sys/kernel/config/target/iser
Allowed WWNs list type: iqn
Fabric module specfile: /var/target/fabric/iser.spec
Fabric module features: discovery_auth, acls, acls_auth, nps, tpgts
Corresponding kernel module: iscsi_target_mod
/>

Upon targetcli initialization, the underlying RTSlib loads the installed fabric modules, and creates the corresponding ConfigFS mount points (at /sys/kernel/config/target/<fabric>), as specified by the associated spec files (located in /var/target/fabric/fabric.spec).

Display the object tree

Use ls to list the object hierarchy, which is initially empty:

/> ls
o- / ..................................................................... [...]
  o- backstores .......................................................... [...]
  | o- fileio ............................................... [0 Storage Object]
  | o- iblock ............................................... [0 Storage Object]
  | o- pscsi ................................................ [0 Storage Object]
  | o- rd_dr ................................................ [0 Storage Object]
  | o- rd_mcp ............................................... [0 Storage Object]
  o- ib_srpt ........................................................ [0 Target]
  o- iscsi .......................................................... [0 Target]
  o- loopback ....................................................... [0 Target]
  o- qla2xxx ........................................................ [0 Target]
/>

Per default, auto_cd_after_create is set to true, which automatically enters an object context (or working directory) after its creation. The examples here are modeled after this behavior.

Optionally, set auto_cd_after_create=false to prevent targetcli from automatically entering new object context after their their creation:

/> set global auto_cd_after_create=false
Parameter auto_cd_after_create is now 'false'.
/>

Create a backstore

Create a backstore using the IBLOCK or FILEIO type devices.

For instance, enter the top-level backstore context and create an IBLOCK backstore from a /dev/sdb block device:

/> cd backstores/
/backstores> iblock/ create name=my_disk dev=/dev/sdb
Generating a wwn serial.
Created iblock storage object my_disk using /dev/sdb.
Entering new node /backstores/iblock/my_disk.
/backstores/iblock/my_disk>

targetcli automatically creates a WWN serial ID for the backstore device and then changes the working context to it.

The resulting object hierarchy looks as follows (displayed from the root object):

/> ls
o- / ..................................................................... [...]
  o- backstores .......................................................... [...]
  | o- fileio ............................................... [0 Storage Object]
  | o- iblock ............................................... [1 Storage Object]
  | | o- my_disk .......................................... [/dev/sdb activated]
  | o- pscsi ................................................ [0 Storage Object]
  | o- rd_dr ................................................ [0 Storage Object]
  | o- rd_mcp ............................................... [0 Storage Object]
  o- ib_srpt ........................................................ [0 Target]
  o- iscsi .......................................................... [0 Target]
  o- loopback ....................................................... [0 Target]
  o- qla2xxx ........................................................ [0 Target]
/>

Alternatively, any LVM logical volume can be used as a backstore, please refer to the LIO Admin Manual on how to create them properly.

For instance, create an IBLOCK backstore on a logical volume (under /dev/<volume_group_name>/<logical_volume_name>):

/backstores> iblock/ create name=my_disk dev=/dev/vg0/lv1
Generating a wwn serial.
Created iblock storage object my_disk using /dev/vg0/lv1.
Entering new node /backstores/iblock/my_disk.
/backstores/iblock/my_disk>

Again, targetcli automatically creates a WWN serial ID for the backstore device and then changes the working context to it.

Instantiate a target

Instantiate an iSER target on the existing backstore my_disk:

/backstores/iblock/my_disk> /iscsi create
Created target iqn.2003-01.org.linux-iscsi.san01.x8664:sn.05135a0e4a11.
Selected TPG Tag 1.
Successfully created TPG 1.
Entering new node /iscsi/iqn.2003-01.org.linux-iscsi.san01.x8664:sn.05135a0e4a11/tpgt1.
/iscsi/iqn.20...a0e4a11/tpgt1>

targetcli automatically creates the Target Portal Group (TPG) and per default assigns a sequentially increasing TPG tag, starting from '1', thereby creating a TPG1.

Export LUNs

Add LUNs to the iSER target:

/iscsi/iqn.20...a0e4a11/tpgt1> luns/ create /backstores/iblock/my_disk
Selected LUN 0.
Successfully created LUN 0.
Entering new node /iscsi/iqn.2003-01.org.linux-iscsi.san01.x8664:sn.05135a0e4a11/tpgt1/luns/lun0.
/iscsi/iqn.20...gt1/luns/lun0>

targetcli per default automatically assigns sequentially increasing LUN IDs, starting from '0', thereby exporting /dev/sdb as LUN 0 in the example above.

Return to the underlying TPG as the working context, as no attributes need to be set or modified for standard LUNs:

/iscsi/iqn.20...gt1/luns/lun0> cd <
Taking you back to /iscsi/iqn.2003-01.org.linux-iscsi.san01.x8664:sn.05135a0e4a11/tpgt1.
/iscsi/iqn.20...a0e4a11/tpgt1>

Create a network portal

Assign an active IP address (here the IPv4 address 192.168.1.139) to the iSER TPG to form a valid iSER Endpoint:

/iscsi/iqn.20...a0e4a11/tpgt1> portals/ create 192.168.1.139
Using default IP port 3260
Successfully created network portal 192.168.1.139:3260.
Entering new node /iscsi/iqn.2003-01.org.linux-iscsi.san01.x8664:sn.05135a0e4a11/tpgt1/portals/192.168.1.139:3260.
/iscsi/iqn.20...68.1.139:3260> ls
o- 192.168.1.139:3260 ......................................... [OK, iser disabled]
/iscsi/iqn.20...68.1.139:3260> cd <

For iSER Network Portals, targetcli automatically uses the iSCSI default port number of '3260', thereby forming a valid new iSER Endpoint. The iSER Endpoint makes the TPG discoverable by iSER initiators.

Enable iSER

/iscsi/iqn.20...68.1.139:3260> iser_enable
iser operation has been enabled
/iscsi/iqn.20....100.0.1:3260> ls
o- 192.168.1.139:3260 .......................................... [OK, iser enabled]
/iscsi/iqn.20...68.1.139:3260> cd <
Taking you back to /iscsi/iqn.2003-01.org.linux-iscsi.san01.x8664:sn.05135a0e4a11/tpgt1.
/iscsi/iqn.20...a0e4a11/tpgt1>

Define access rights

Configure the access rights to allow logins from initiators. The three basic setups for LUN authentication are discussed below.

Demo mode

For simple demo setups, "open" access can be granted to all initiators so that they can access all LUNs in the TPG without further authentication. To enable that so-called "demo mode" TPG operation, disable all authentication for the corresponding Endpoint:

/iscsi/iqn.20...a0e4a11/tpgt1> set attribute authentication=0 demo_mode_write_protect=0
generate_node_acls=1 cache_dynamic_acls=1.
Parameter demo_mode_write_protect is now '0'.
Parameter authentication is now '0'.
Parameter generate_node_acls is now '1'.
Parameter cache_dynamic_acls is now '1'.
/iscsi/iqn.20...a0e4a11/tpgt1> cd /
/>

This exports the IBLOCK backstore as LUN0 to initiators without any access restrictions.

Use "demo mode" only under the following conditions:

CHAP initiator authentication

Enable secure sessions for the initiator with the IQN "iqn.1991-05.com.microsoft:ibm-t410s":

/iscsi/iqn.20...a0e4a11/tpgt1> acls/ create iqn.1991-05.com.microsoft:ibm-t410s
Successfully created Node ACL for iqn.1991-05.com.microsoft:ibm-t410s
Created mapped LUN 0.
Entering new node /iscsi/iqn.2003-01.org.linux-iscsi.san01.x8664:sn.05135a0e4a11/tpgt1/acls/iqn.1991-05.com.microsoft:ibm-t410s/mapped_lun0.
/iscsi/iqn.20...s/mapped_lun0> cd <
Taking you back to /iscsi/iqn.2003-01.org.linux-iscsi.san01.x8664:sn.05135a0e4a11/tpgt1.
/iscsi/iqn.20...a0e4a11/tpgt1>

This creates an iSER Node ACL with a mapped LUN0.

Node ACLs allow mappings of actual LUN IDs onto arbitrary Mapped_LUN IDs, which are the LUN IDs presented to initiators. These mappings can match preferred LUN IDs for particular initiators, so for instance, a LUN1 can be mapped onto Mapped_LUN0 to make LUN1 appear as iSER LUN0 on the initiator. Usually, LUNs are identically mapped, however.

Setup the CHAP logon information for an initiator, which consists of the userid (login name) and password (target secret) from the initiator:

/iscsi/iqn.20...a0e4a11/tpgt1> cd acls/iqn.1991-05.com.microsoft:ibm-t410s
/iscsi/iqn.20...oft:ibm-t410s> set auth userid=iqn.1991-05.com.microsoft:ibm-t410s
Parameter userid is now 'iqn.1991-05.com.microsoft:ibm-t410s'.
/iscsi/iqn.20...oft:ibm-t410s> set auth password=mytargetsecret
Parameter password is now 'mytargetsecret'.
/iscsi/iqn.20...oft:ibm-t410s> get auth
AUTH CONFIG GROUP
  mutual_password=
    The mutual_password auth parameter.

  mutual_userid=
    The mutual_userid auth parameter.

  password=mytargetsecret
    The password auth parameter.

  userid=iqn.1991-05.com.microsoft:ibm-t410
    The userid auth parameter.
/iscsi/iqn.20...oft:ibm-t410s> cd /iscsi
/iscsi>

The iSER Endpoint is now ready for secure logins from the specified iSER initiator.

Mutual CHAP authentication

Enable secure sessions for the initiator with the IQN "iqn.1991-05.com.microsoft:ibm-t410s":

/iscsi/iqn.20...a0e4a11/tpgt1> acls/ create iqn.1991-05.com.microsoft:ibm-t410s
Successfully created Node ACL for iqn.1991-05.com.microsoft:ibm-t410s
Created mapped LUN 0.
Entering new node /iscsi/iqn.2003-01.org.linux-iscsi.san01.x8664:sn.05135a0e4a11/tpgt1/acls/iqn.1991-05.com.microsoft:ibm-t410s/mapped_lun0.
/iscsi/iqn.20...s/mapped_lun0> cd <
Taking you back to /iscsi/iqn.2003-01.org.linux-iscsi.san01.x8664:sn.05135a0e4a11/tpgt1.
/iscsi/iqn.20...a0e4a11/tpgt1>

This creates an iSER Node ACL with a mapped LUN0.

Node ACLs allow mappings of actual LUN IDs onto arbitrary Mapped_LUN IDs, which are the LUN IDs presented to initiators. These mappings can match preferred LUN IDs for particular initiators, so for instance, a LUN1 can be mapped onto Mapped_LUN0 to make LUN1 appear as iSER LUN0 on the initiator. Usually, LUNs are identically mapped, however.

Setup the mutual CHAP logon information for an initiator, which consists of:

/iscsi/iqn.20...a0e4a11/tpgt1> cd acls/iqn.1991-05.com.microsoft:ibm-t410s
/iscsi/iqn.20...oft:ibm-t410s> set auth userid=iqn.1991-05.com.microsoft:ibm-t410s password=mytargetsecret mutual_userid=iqn.2003-01.org.linux-iscsi.san01.x8664:sn.05135a0e4a11 mutual_password=mymutualsecret
Parameter userid is now 'iqn.1991-05.com.microsoft:ibm-t410s'.
Parameter password is now 'mytargetsecret'.
Parameter mutual_userid is now 'iqn.2003-01.org.linux-iscsi.san01.x8664:sn.05135a0e4a11'.
Parameter password is now 'mymutualsecret'.
/iscsi/iqn.20...oft:ibm-t410s> get auth
AUTH CONFIG GROUP
  mutual_password=mymutualsecret
    The mutual_password auth parameter.

  mutual_userid=iqn.2003-01.org.linux-iscsi.san01.x8664:sn.05135a0e4a11
    The mutual_userid auth parameter.

  password=mytargetsecret
    The password auth parameter.

  userid=iqn.1991-05.com.microsoft:ibm-t410
    The userid auth parameter.
/iscsi/iqn.20...oft:ibm-t410s> cd /iscsi
/iscsi>

The iSER Endpoint is now ready for secure logins from the specified iSER initiator.

TPG authentication

Setting up authentication information for every single initiator separately can be cumbersome, so targetcli provides the capability to define common login information for all Endpoints in a TPG. As a result, all initiators connecting to that TPG can use the same login credentials.

Enable TPG authentication for all Endpoints in a TPG:

/iscsi/iqn.20...a0e4a11/tpgt1> /iscsi/iqn.2003-01.org.linuxiscsi.
san01.x8664:sn.bf919196ff4e/tgpt1/ set attribute demo_mode_write_protect=0 generate_node_acls=1
cache_dynamic_acls=1
Parameter demo_mode_write_protect is now '0'.
Parameter generate_node_acls is now '1'.
Parameter cache_dynamic_acls is now '1'.
/iscsi/iqn.20...a0e4a11/tpgt1>

Setup the the common TPG authentication credentials for all Endpoints in a TPG, which consists of:

/iscsi/iqn.20...a0e4a11/tpgt1> set auth userid=rts-user
Parameter userid is now 'rts-user'.
/iscsi/iqn.20...a0e4a11/tpgt1> set auth password=b492785e-bc91-4710
Parameter password is now 'b492785e-bc91-4710'.
/iscsi/iqn.20...a0e4a11/tpgt1> set auth userid_mutual=mutual-rts-user
Parameter userid_mutual is now 'mutual-rts-user'.
/iscsi/iqn.20...a0e4a11/tpgt1> set auth password_mutual=aeae2e26-f043-42a7
Parameter password_mutual is now 'aeae2e26-f043-42a7'.
/iscsi/iqn.20...a0e4a11/tpgt1> get auth
AUTH CONFIG GROUP
  authenticate_target=0 [ro]
    The authenticate_target auth_attr.

  password=b492785e-bc91-4710
    The password auth_attr.

  password_mutual=aeae2e26-f043-42a7
    The password_mutual auth_attr.

  userid=rts-user
    The userid auth_attr.

  userid_mutual=mutual-rts-user
    The userid_mutual auth_attr.
/iscsi/iqn.20...a0e4a11/tpgt1>

Enable discovery control

Optionally, the iSER protocol can also control the visibility of iSER targets for discovery by iSER initiators.

CHAP initiator discovery authentication

Enable CHAP initiator discovery authentication for all initiators by setting up a CHAP userid (login name) and password (target secret) in the global discovery_auth group:

/iscsi> set discovery_auth enable=1 userid=mytargetuid password=mytargetsecret
Parameter enable is now '1'.
Parameter password is now 'mytargetsecret'.
Parameter userid is now 'mytargetuid'.
/iscsi>

Only iSER initiators that can authenticate themselves with a user id of "mytargetuid" and a password of "mytargetsecret" can now discover this iSER target.

Mutual CHAP discovery authentication

Enable Mutual CHAP discovery authentication for all initiators by setting up Mutual CHAP information in the global discovery_auth group:

/iscsi> set discovery_auth enable=1 userid=mytargetuid password=mytargetsecret
mutual_userid=mymutualuid mutual_password=mymutualsecret
Parameter password is now 'mytargetsecret'.
Parameter userid is now 'mytargetuid'.
Parameter mutual_password is now 'mymutualsecret'.
Parameter mutual_userid is now 'mymutualuid'.
Parameter enable is now '1'.
/iscsi> get discovery_auth
DISCOVERY_AUTH CONFIG GROUP
  enable=1
    The enable discovery_auth parameter.

  mutual_password=mymutualsecret
    The mutual_password discovery_auth parameter.

  mutual_userid=mymutualuid
    The mutual_userid discovery_auth parameter.

  password=mytargetsecret
    The password discovery_auth parameter.

  userid=mytargetuid
    The userid discovery_auth parameter.
/iscsi>

Only iSER initiators that can authenticate themselves with a user id of "mytargetuid" and a password of "mytargetsecret" can now discover this iSER target, and conversely, the iSER target can only discover iSER initiators that can authenticate themselves with a user id of "mymutualuid" and a password of "mymutualsecret".

Display the object tree

The resulting iSCSI object hierarchy with CHAP Authentication (initiator or mutual) for one initiator looks as follows (displayed from the root object):

/> ls
o- / ..................................................................... [...]
  o- backstores .......................................................... [...]
  | o- fileio ............................................... [0 Storage Object]
  | o- iblock ............................................... [1 Storage Object]
  | | o- my_disk .......................................... [/dev/sdb activated]
  | o- pscsi ................................................ [0 Storage Object]
  | o- rd_dr ................................................ [0 Storage Object]
  | o- rd_mcp ............................................... [0 Storage Object]
  o- ib_srpt ........................................................ [0 Target]
  o- iscsi .......................................................... [1 Target]
  | o- iqn.2003-01.org.linux-iscsi.san01.x8664:sn.05135a0e4a11 ......... [1 TPG]
  |   o- tpgt1 ....................................................... [enabled]
  |     o- acls ........................................................ [1 ACL]
  |     | o- iqn.1991-05.com.microsoft:ibm-t410s ................ [1 Mapped LUN]
  |     |   o- mapped_lun0 ......................................... [lun0 (rw)]
  |     o- luns ........................................................ [1 LUN]
  |     | o- lun0 .................................. [iblock/my_disk (/dev/sdb)]
  |     o- portals .................................................. [1 Portal]
  |       o- 192.168.1.139:3260 ............................. [OK, iser enabled]
  o- loopback ....................................................... [0 Target]
  o- qla2xxx ........................................................ [0 Target]
/>

Persist the configuration

Use saveconfig from the root context to persist the target configuration across OS reboots:

/> saveconfig
WARNING: Saving rtsnode1 current configuration to disk will overwrite your boot settings.
The current target configuration will become the default boot config.
Are you sure? Type 'yes': yes
Making backup of srpt/ConfigFS with timestamp: 2012-02-27_23:19:37.660264
Successfully updated default config /etc/target/srpt_start.sh
Making backup of qla2xxx/ConfigFS with timestamp: 2012-02-27_23:19:37.660264
Successfully updated default config /etc/target/qla2xxx_start.sh
Making backup of loopback/ConfigFS with timestamp: 2012-02-27_23:19:37.660264
Successfully updated default config /etc/target/loopback_start.sh
Making backup of LIO-Target/ConfigFS with timestamp: 2012-02-27_23:19:37.660264
Successfully updated default config /etc/target/lio_backup-2012-02-27_23:19:37.660264.sh
Making backup of Target_Core_Mod/ConfigFS with timestamp: 2012-02-27_23:19:37.660264
Successfully updated default config /etc/target/tcm_backup-2012-02-27_23:19:37.660264.sh
Generated Target_Core_Mod config: /etc/target/backup/tcm_backup-2012-02-27_23:19:37.660264.sh
Successfully updated default config /etc/target/lio_start.sh
Successfully updated default config /etc/target/tcm_start.sh
/>

Spec file

Datera spec files define the fabric-dependent feature set, capabilities and available target ports of the specific underlying fabric.

In particular, iSER uses the iSCSI spec file, and does not require a seperate spec file.

Scripting with RTSlib

Setup script

The following Python code illustrates how to setup a basic iSER target and export a mapped LUN:

#!/usr/bin/python
# iSER setup script example with RTSlib
from rtslib import *

# Setup an IBLOCK backstore
backstore = IBlockBackstore(3, mode='create')
try:
    so = IBlockStorageObject(backstore, "sdb", "/dev/sdb", gen_wwn=True)
except:
    backstore.delete()
    raise

# Create an iSER target endpoint using an iSCSI IQN
fabric = FabricModule('iSCSI')
target = Target(fabric, "iqn.2003-01.org.linux-iscsi.x.x8664:sn.d3d8b0500fde")
tpg = TPG(target, 1)

# Setup a network portal in the iSER TPG
# The IP address must already be active on the system
portal = NetworkPortal(tpg, "192.168.1.128", "5060")

# Enable iSER for the portal (only difference to iSCSI)
portal._set_iser_attr(1)

# Export LUN 0 via the 'so' StorageObject class
lun0 = tpg.lun(0, so, "my_lun")

# Setup the NodeACL for an iSER initiator, and create MappedLUN 0
node_acl = tpg.node_acl("iqn.2003-01.org.linux-iscsi.y.x8664:sn.abcdefghijkl")
mapped_lun = node_acl.mapped_lun(0, 0, False)

Object tree

The resulting object tree looks as follows:

o- / ..................................................................... [...]
  o- backstores .......................................................... [...]
  | o- fileio ............................................... [0 Storage Object]
  | o- iblock ............................................... [1 Storage Object]
  | | o- sdb .............................................. [/dev/sdb activated]
  | o- pscsi ................................................ [0 Storage Object]
  | o- rd_dr ................................................ [0 Storage Object]
  | o- rd_mcp ............................................... [0 Storage Object]
  o- ib_srpt ........................................................ [0 Target]
  o- iscsi .......................................................... [1 Target]
  | o- iqn.2003-01.org.linux-iscsi.x.x8664:sn.d3d8b0500fde ............. [1 TPG]
  |   o- tpgt1 ....................................................... [enabled]
  |     o- acls ........................................................ [1 ACL]
  |     | o- iqn.2003-01.org.linux-iscsi.y.x8664:sn.abcdefghijkl. [1 Mapped LUN]
  |     |   o- mapped_lun0 ......................................... [lun0 (rw)]
  |     o- luns ........................................................ [1 LUN]
  |     | o- lun0 ...................................... [iblock/sdb (/dev/sdb)]
  |     o- portals .................................................. [1 Portal]
  |       o- 192.168.1.128:5060 ............................. [OK, iser enabled]
  o- loopback ....................................................... [0 Target]
  o- qla2xxx ........................................................ [0 Target]

RFCs

See also

Notes

  1. Linus Torvalds (2013-6-30). "Linux 3.10". marc.org. 
  2. "Open Consortium Developing Specifications for Remote Direct Memory Access Over TCP/IP Networks". press release. 2002-05-31. 
  3. Mike Ko, et al. (July 2003). "iSCSI Extensions for RDMA Specification (Version 1.0)". 
  4. M. Ko et al. (October 2007). "Internet Small Computer System Interface (iSCSI) Extensions for Remote Direct Memory Access (RDMA)". RFC 5046. 
  5. M. Chadalapaka et al. (October 2007). "DA: Datamover Architecture for the Internet Small Computer System Interface (iSCSI)". RFC 5047. 
  6. "OpenFabrics Alliance". official web site. 

Wikipedia entries

External links

Timeline of the LinuxIO
Release Details 2011 2012 2013 2014 2015
123456789101112 123456789101112 123456789101112 123456789101112 123456789101112
4.x Version 4.0 4.1
Feature LIO Core Loop back FCoE iSCSI Perf SRP
CM WQ FC
USB
1394
vHost Perf Misc 16 GFC iSER Misc VAAI Misc DIF Core
NPIV
DIF iSER DIF FC vhost TCMU Xen Misc Misc virtio 1.0 Misc NVMe OF
Linux 2.6.38 2.6.39 3.0 3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9 3.10 3.11 3.12 3.13 3.14 3.15 3.16 3.17 3.18 3.19 3.20 3.21 3.22
Personal tools
Namespaces
Variants
Actions
Navigation
Toolbox
Google AdSense