This text assumes that the reader is familiar with Ethernet technology such as a packet, a MAC address an IP address. If you do not have this knowledge it is not a no-go but it will facilitate things when comparing Ethernet to Fiberchannel as a network system.
Addressing in Fiberchannel
In a Fiberchannel network, each network node has a World Wide Node Number (WWNN). Every network node has one or more World Wide Port Numbers (WWPN).
Imagine a street where many buildings have a house number. There can be some little houses for one familiy having one entrance as well as a big company having a house number but multiple entrances. In the latter you could drive to "XY corporation, Example Drive 38, Entrance D". 38 is the house number - or the World Wide Node Number. "D" would represent the entrance or the "World Wide Port Number".
The WWPN (World Wide Port Number) is an absolute and unique address - the world wide node address may have many ports assigned to it. Communications take place between WWPNs. Normally, WWPNs may be derived from the WWNN. Look at the following graphic which shows a very easy configuration: (there are some vendors where the WWPN may not be derived from the WWNN of the device, this is a very simple example)
You see a storage device offering four connection ports - like a machine with four ethernet network interfaces. The device has four distinct fiberchannel interfaces and each of them has its unique World Wide Port Number (WWPN). Think of the WWPN like a MAC address on an ethernet network. It is unique. The WWNN (World Wide Node Number) is also unique - but for the device. The distinction between WWNNs and WWPNs is here in the easy example the first byte of the network address:
20 - this is a Node (WWNN)
21 - this is the WWPN of Port 1
22 - this is the WWPN of Port 2
With a normal computer as fiberchannel host things are a little more difficult. If you buy a Fiberchannel interface card (think of it as a network device like an ethernet interface card) you'll get a WWNN which is unique per interface - but each port of the interface card will have its WWPN like mentioned above. So if you've bought a single-port Fiberchannel card, it will have a WWPN (Port Number) starting with hex "21", and if you've got a dual-port card it will have WWPNs whith 21 and 22:
The World Wide Node Number (WWNN) and the World Wide Port Number (WWPN) are both a World Wide Number (WWN) in the Fiberchannel Protocol.
Each WWN is 64 bit long - 8 Bytes. On Ethernet a MAC-address has 48 bits and each ethernet port has its own MAC address - the WWPN is the MAC address for fiberchannel networks. Ethernet does not know something comparable to a World Wide Node Address.
We've learned now the difference between a WWNN and a WWPN. This is fundamental in Fiberchannel's design.
But now we have to form a network with our WWNNs so that the WWPNs may talk to each other.
Network topologies
The simpliest way to interconnect fiberchannel devices is shown on the right: Just plug an optical cable (may also be a copper connection, depends on the type of fiberchannel interface you have installed) between the two machines you want to talk to each other.
This is done sometimes when a large storage array has to be connected to the host wanting to use it exclusively - like a big external hard disk.
You see there's marked "L_port". The fiberchannel interface has to be in an L_port-configuration (Loop Port) - as well as the interface of your storage array on the other end.
In fact this is the simpliest case of an
Arbitrated Loop.
Arbitrated Loops (ALs) have been used extensively in the early days of fiberchannel networking - nowadays it has disappeared nearly completely - except for this "back-to-back" configuration.
In an arbitrated loop devices are daisy-chained, just like in the early parallel SCSI days - with the exception that you don't need a termination here. The devices are confguring their ports automatically (they arbitrate the loop themselves).
An arbitrated loop fiberchannel network can be connected like this:
Nowadays normally we use a switch fabric network. That's the same thing you do with ethernet networks using switches to connect hosts to the network. In Fiberchannel terminology this is called a
Switched Fabric.
For a fabric network we need N_ports and F_ports. If you recall ethernet you remember that you needed a twisted cable to be able to connect switches to each other (nowadays most gigabit switch ports are completely autosensing) and with Fiberchannel we have something similar: The ports are not the same. In fact we have 3 primary port types and 2 secondary ones:
L_port
This is the port type we need to form an arbitrated loop. L_ports are autoconfigured and need to know the topology of the loop.
N_port
This is the node port. Every fiberchannel node (a fiberchannel card, a storage device, a fiberchannel tape, ...) is an N_port as default. You can not connect two N_ports to talk to each other.
F_port
This is a port of a fiberchannel switch, it is named a fabric port. You can connect two F_ports together to create an interswitch-link.
The two secondary port types are:
NL_port
This is a normal node port which is capable to connect to an arbitrated loop as well - it has loop capability.
FL_port
This is a switch port which can emulate a loop port. If a switch port is configured as FL_port, you may use a daisy chain of loop devices on this port.
The fabric configuration has a massive disadvantage however: If the switch on the left fails you won't have any access to the storage device any more. The switch is a single point of failure in this scenario.
Typically you use a redundant configuration when using a switched fabric.
This is an example:
The "red" and "yellow" networks do not see each other. On one fiberchannel interface the hosts sees the red interfaces of the storage devices, the other interface sees the yellow network.
If one switch breaks, you still have connection.
Problem: In this scenario, the host computer will see four storage devices instead of two, because it does not know that two different links lead to the same device. To cover this you need multipathing. More on this in part 2.
Often you don't want to have devices seeing each other. On Ethernet LANs you can use VLANs to dinstinguish multiple networks and you use trunks to carry multiple networks across the switch structure.
On Fiberchannel this is more or less the same with the difference that the networks are named "Zones" (Cisco tries to rename that to "VSAN" to have something similar to VLAN):
The trunk connection on the right carries information for both zones to another switch where the same zones can be made available this way.
Protocols
Now let's talk about protocols we can "talk" with our fiberchannel network.
The most important used protocol is just named "fcp", it is used to speak SCSI to the storage devices. This makes up a SAN.
But you can talk IP along with UDP and TCP as well - this is named "fcip". Care is needed when designing firewall structures - sometimes people forget that IP communication is easily done via fiberchannel as well. Proper Zoning is essential in these configurations to prevent this.
Fiberchannel target / initiator terminology
Using fiberchannel storage devices there are always two sides of communication (on the protocol layer):
The fiberchannel initiator initiates the connect and requests storage from the storage device. A normal hosts wanting to use a fiberchannel storage device will be the initiator.
The fiberchannel target accepts fiberchannel SCSI commands and executes them - the storage device has to be in target mode.
In Solaris, the fiberchannel target mode for zfs volumes is the missing gap. iSCSI target and initiator are available but fiberchannel is only supported as initiator.
No comments:
Post a Comment