Our PS6000XV models are used for our Enterprise client production storage pools, and our E-VS Citrix XenServer pools. Our PS4000 usage is primarily for offsite asynchronous replication which we provide to all of our Enterprise customers.
Today is an excellent example of the Equallogic platform handling an event failure, dispatching notifications and self-healing.
Disk 7 of the array unit went into a failure status, the Equallogic group immediately replaced this disk with a hot spare from the member array (each Equallogic keeps 2 hot spares per chassis). Immediately following began a rebuild of the RAID-10 degradation and an email report to our Operations and the Dell Equallogic support group.
Keeping mission critical support services on our Equallogic PS6000XVÂÂs DellÂÂs internal Operations team had a ticket opened before we were able to call in and dispatch a request for resolution.
Here are before and after pictures of the array, as we all love to look at pictures.
High Availability, shortened as HA more often than not, is the concept of N+1 or greater service availability in a datacenter/computing environment. This post will begin a series on High Availability related to Network Redux infrastructure.
One basic, yet underestimated component to the stack is the distribution layer. In our world the distribution layer often participates in a collapsed core environment, distributing network packets downstream to a pair of clustered firewall appliances.
Within the distribution layer we utilize the open standard Virtual Router Redundancy Protocol. For Cisco Engineers you will also have the Cisco flavor with HSRP.
VRRP is a powerful tool in the HA war chest for system architects. From a 30,000 ft. review, it provides a floating default gateway address between two active routers in a network. Though an important point to consider, from a layer-1/layer-2 perspective these are both active and viable paths, hence you may see egress traffic leave your network over the secondary/backup interface. This is more common than not in my experience.
In our network VRRP is delivered via our Top of the Rack (ToR) Force10 Switches. Here is a sample configuration for what we will call sr-0-1 and sr-0-2, where sr-0-1 will be acting as the primary/master. For posterity I have chosen an unused public network subnet within our network.
interface Vlan 100 description client-net-68-233-0-0-24 ip address 184.108.40.206/24 tagged Port-channel 1 untagged GigabitEthernet 2/43 ! vrrp-group 100 description client-vrrp-68-233-0-0-24 virtual-address 220.127.116.11 priority 110 no shutdown !
interface Vlan 100 description client-net-68-233-0-0-24 ip address 18.104.22.168/24 tagged Port-channel 1 untagged GigabitEthernet 2/43 ! vrrp-group 100 description client-vrrp-68-233-0-0-24 virtual-address 22.214.171.124 no shutdown !
A couple of assumptions are made in this design:
1) Both downstream interfaces are connected to interface 2/43 on each respective switch/router.
2) 126.96.36.199/24 is a routed and recognized network within this architecture.
3) A port-channel of some form exists for communication to take place between sr-0-1 and sr-0-2. Broken communication would result in both sides indicating themselves to be master.
Lets break down some of the segments for a deeper understanding as to what these instructions will accomplish:
1) We provide vlan 100 with an IP address in the network segment for communication between these routed interfaces. Some will argue VRRP is not a layer-3 protocol, the happy medium is to call it a protocol between Layer2/3, it doesnt really have a formal home.
2) virtual IP address is the floating gateway that will be used by the downstream devices as their default gateway.
3) description nomenclature is just based on best practices at Network Redux.
4) priority 110 will provide that interface with a higher priority as the default priority is 100. If we wanted to delegate sr-0-2 to master a simply priority 90 command would adjust this setting. VRRP in this type of design is so fast it would barely, if at all be noticed as a failover.
With this configuration live, the following would be seen from a show vrrp brief command:
Interface Grp Pri Pre State Master addr Virtual addr(s) Description -------------------------------------------------------------------- Vl 100 100 110 Y Master 188.8.131.52 184.108.40.206 client-vrrp-68-233-0-0-24
Interface Grp Pri Pre State Master addr Virtual addr(s) Description -------------------------------------------------------------------- Vl 100 100 100 Y Backup 220.127.116.11 18.104.22.168 client-vrrp-68-233-0-0-24
And there you have it, a redundant distribution layer. In our world we bypass the access layer and distribute directly to our servers or security appliances. Most common would be a set of redundant firewalls in active/passive mode.
This configuration syntax is specific to the Force10 Operating System (FTOS), but VRRP as a protocol is open and widely used across all switching/routing vendors in the industry.