Red Hat GFS 6.0: Administrator's Guide
PrevChapter 6. Creating the Cluster Configuration System FilesNext

6.6. Creating the fence.ccs File

You can configure each node in a GFS cluster for a variety of fencing devices. To configure fencing for a node, you need to perform the following tasks:

Creating the fence.ccs file consists of defining each fencing device you are going to use. You can define the following types of fencing devices in the fence.ccs file:

The fence.ccs file is used in conjunction with the nodes.ccs file to configure fencing in a cluster. The nodes.ccs file specifies fencing devices that are defined in the fence.ccs file. The fence.ccs file can define any combination of fencing devices.

If a node has dual power supplies, you must define a fencing device for each power supply. Similarly, if a node has multiple paths to FC storage, you must define a fencing device for each path to FC storage.

For more information about fencing, refer to Chapter 10 Using the Fencing System.

To create the fence.ccs file, follow these steps:

  1. Create a file named fence.ccs. Use a file format according to the fencing method as follows. Refer to Table 6-2 for syntax description.

    • APC MasterSwitch — Refer to Figure 6-2.

    • WTI NPS (Network Power Switch) — Refer to Figure 6-3.

    • Brocade FC (Fibre Channel) switch — Refer to Figure 6-4.

    • McData FC (Fibre Channel) switch — Refer to Figure 6-5.

    • Vixel FC switch — Refer to Figure 6-6.

    • GNBD — For GNBD without GNBD multipath, refer to Figure 6-7. For GNBD with GNBD multipath, refer to Figure 6-8.

    • HP RILOE card — Refer to Figure 6-9.

    • xCAT — Refer to Figure 6-10.

    • Egenera BladeFrame system — Refer to Figure 6-11.

    • Manual — Refer to Figure 6-12.

      WarningWarning
       

      Manual fencing should not be used in a production environment. Manual fencing depends on human intervention whenever a node needs recovery. Cluster operation is halted during the intervention.

  2. Type parameters in the file according to the fencing device (or devices) needed:

    1. For each APC MasterSwitch fencing device, specify the following parameters: DeviceName, the fencing agent (agent =) as fence_apc, IPAddress, LoginName, and LoginPassword. Refer to Example 6-2 for a fence.ccs file that specifies an APC MasterSwitch fencing device.

    2. For each WTI NPS fencing device, specify the following parameters: DeviceName, the fencing agent (agent =) as fence_wti, IPAddress, and LoginPassword. Refer to Example 6-3 for a fence.ccs file that specifies a WTI NPS fencing device.

    3. For each Brocade FC-switch fencing device, specify the following parameters: DeviceName, the fencing agent (agent =) as fence_brocade, IPAddress, LoginName, and LoginPassword. Refer to Example 6-4 for a fence.ccs file that specifies a Brocade FC-switch fencing device.

    4. For each McData FC-switch fencing device, specify the following parameters: DeviceName, the fencing agent (agent =) as fence_mcdata, IPAddress, LoginName, and LoginPassword. Refer to Example 6-5 for a fence.ccs file that specifies a McData FC-switch fencing device.

    5. For each Vixel FC-switch fencing device, specify the following parameters: DeviceName, the fencing agent (agent =) as fence_vixel, IPAddress, and LoginPassword. Refer to Example 6-6 for a fence.ccs file that specifies a Vixel FC-switch fencing device.

    6. For each GNBD fencing device, specify the following parameters: DeviceName, the fencing agent (agent =) as fence_gnbd, and ServerName.

      For GNBD multipath, include an option = "multipath" line after the ServerName line. In addition, for GNBD multipath, you can add two optional lines: retrys = "Number" and wait_time = "Seconds".

      NoteNote
       

      Do not use fence_gnbd to fence GNBD server nodes.

      For descriptions of those parameters refer to Table 6-2. Refer to Example 6-7 for a fence.ccs file that specifies a GNBD fencing device for a configuration that does not employ GNBD multipath. Refer to Example 6-8 for a fence.ccs file that specifies a GNBD fencing device for a configuration that does employ GNBD multipath.

    7. For each HP-RILOE-card fencing device, specify the following parameters: DeviceName, the fencing agent (agent =) as fence_rib, HostName, LoginName, and LoginPassword. Refer to Example 6-9 for a fence.ccs file that specifies an HP-RILOE-card fencing device.

    8. For each xCAT fencing device, specify the following parameters: DeviceName, the fencing agent (agent =) as fence_xcat, and RpowerBinaryPath. Refer to Example 6-10 for a fence.ccs file that specifies an xCAT fencing device.

    9. For each Egenera BladeFrame fencing device, specify the following parameters: DeviceName, the fencing agent (agent =) as fence_egenera, and CserverName. Refer to Example 6-11 for a fence.ccs file that specifies an Egenera BladeFrame fencing device.

    10. For each manual fencing device, specify DeviceName and the fencing agent (agent =) as fence_manual. Refer to Example 6-12 for a fence.ccs file that specifies a manual fencing device.

      WarningWarning
       

      Manual fencing should not be used in a production environment. Manual fencing depends on human intervention whenever a node needs recovery. Cluster operation is halted during the intervention.

  3. Save the file.

fence_devices{ 
   DeviceName { 
      agent = "fence_apc" 
      ipaddr = "IPAddress" 
      login = "LoginName" 
      passwd = "LoginPassword" 
   } 
   DeviceName {
   .
   .
   .
   } 
}

Figure 6-2. File Structure: fence_devices, fence_apc

fence_devices{ 
   DeviceName { 
      agent = "fence_wti" 
      ipaddr = " IPAddress" 
      passwd = " LoginPassword" 
   } 
   DeviceName { 
   .
   .
   .
   } 
}

Figure 6-3. File Structure: fence_devices, fence_wti

fence_devices{ 
   DeviceName { 
      agent = "fence_brocade" 
      ipaddr = "IPAddress" 
      login = "LoginName"
      passwd = "LoginPassword" 
   } 
   DeviceName { 
   .
   .
   .
   } 
}

Figure 6-4. File Structure: fence_devices, fence_brocade

fence_devices{ 
   DeviceName { 
      agent = "fence_mcdata" 
      ipaddr = "IPAddress"
      login = "LoginName" 
      passwd = "LoginPassword" 
   } 
   DeviceName { 
   .
   .
   .
   } 
}

Figure 6-5. File Structure: fence_devices, fence_mcdata

fence_devices{ 
   DeviceName { 
      agent = "fence_vixel" 
      ipaddr = "IPAddress" 
      passwd = "LoginPassword" 
   } 
   DeviceName { 
   .
   .
   .
   } 
}

Figure 6-6. File Structure: fence_devices, fence_vixel

fence_devices{ 
   DeviceName { 
      agent = "fence_gnbd" 
      server = "ServerName"
      .
      .
      .
      server = "ServerName" 
   } 
   DeviceName { 
   .
   .
   .
   } 
}

Figure 6-7. File Structure: fence_devices, fence_gnbd without GNBD Multipath

fence_devices{ 
   DeviceName { 
      agent = "fence_gnbd" 
      server = "ServerName"
      .
      .
      .
      server = "ServerName" 
      option = "multipath"
      retrys = "Number"
      wait_time = "Seconds"
   } 
   DeviceName { 
   .
   .
   .
   } 
}

Figure 6-8. File Structure: fence_devices, fence_gnbd with GNBD Multipath

fence_devices{ 
   DeviceName { 
      agent = "fence_rib" 
      hostname = "HostName"
      login = "LoginName"
      passwd = "LoginPassword"    
   }
   DeviceName { 
   .
   .
   .
   } 
}

Figure 6-9. File Structure: fence_devices, fence_rib

fence_devices{ 
   DeviceName { 
      agent = "fence_xcat" 
      rpower = "RpowerBinaryPath"  
   } 
   DeviceName { 
   .
   .
   .
   } 
}

Figure 6-10. File Structure: fence_devices, fence_xcat

fence_devices{ 
   DeviceName { 
      agent = "fence_egenera" 
      cserver = "CserverName"  
   } 
   DeviceName { 
   .
   .
   .
   } 
}

Figure 6-11. File Structure: fence_devices, fence_egenera

fence_devices{ 
   DeviceName { 
      agent = "fence_manual" 
   }
   DeviceName { 
   .
   .
   .
   } 
}

Figure 6-12. File Structure: fence_devices, fence_manual

WarningWarning
 

Manual fencing should not be used in a production environment. Manual fencing depends on human intervention whenever a node needs recovery. Cluster operation is halted during the intervention.

ParameterDescription
CserverName For Egenera BladeFrame fencing device: The name of an Egenera control blade, the Egenera control blade with which the fence agent communicates via ssh.
DeviceName The name of a fencing device. The DeviceName parameter specifies the name of a fencing device and makes that fencing device available for use by the fence section of a nodes.ccs file (nodes.ccs:NodeName/fence/MethodName). The fence section of a nodes.ccs file also contains DeviceName parameters each mapping to a DeviceName in the fence.ccs file.
HostNameThe host name of a RILOE card on the network to which stunnel connections can be made.
IPAddressFor fencing with power and Fibre Channel switches: The IP address of a switch to which Telnet connections can be established.
LoginNameThe login name for a power switch, an FC switch, or a RILOE card.
LoginPasswordThe password for logging in to a power switch, an FC switch, or a RILOE card.
multipath

Selects GNBD multipath style fencing.
CAUTION: When multipath style fencing is used, if the gnbd_servd process of a GNBD server node cannot be contacted, it is fenced as well, using its specified fencing method. That means that when a GNBD client (GFS node) is fenced, any node listed as its GNBD server that does not have the gnbd_serv module loaded (which starts gnbd_servd) is also fenced.

RpowerBinaryPathFor xCAT fencing device, the path to the rpower binary.
NumberThe number of times to retry connecting to the GNBD server after a failed attempt, before the server is fenced. The parameter entry is for the retrys = entry and is only valid when used with multipath style fencing. (Refer to the multipath entry in this table.) The default value of Number is 3.
SecondsThe length of time, in seconds, to wait between retries. This parameter entry is for the wait_time = entry and is only valid when used with multipath style fencing. (Refer to the multipath entry in this table.) The default value of Seconds is 2.
ServerNameThe host name of a GNBD server. Each GNBD server is represented with a "server =" line in the fence.ccs file. For example, if you have three GNBD servers, then the fence.ccs file needs three "server =" lines one for each GNBD server.

Table 6-2. File Syntax Description: Variables for fence.ccs

fence_devices {  
         apc1 { 
               agent = "fence_apc" 
               ipaddr = "10.0.3.3" 
               login = "apc" 
               passwd = "apc" 
         } 
         apc2 { 
               agent = "fence_apc" 
               ipaddr = "10.0.3.4" 
               login = "apc" 
               passwd = "apc" 
         } 
}

Example 6-2. APC MasterSwitch Fencing Devices Named apc1 and apc2

fence_devices {  
         wti1 { 
               agent = "fence_wti" 
               ipaddr = "10.0.3.3" 
               passwd = "password" 
         } 
         wti2 { 
               agent = "fence_wti" 
               ipaddr = "10.0.3.4" 
               passwd = "password" 
         } 
}

Example 6-3. WTI NPS Fencing Devices Named wti1 and wti2

fence_devices {  
         silkworm1 { 
               agent = "fence_brocade" 
               ipaddr = "10.0.3.3" 
               login = "admin" 
               passwd = "password" 
         } 
         silkworm2 { 
               agent = "fence_brocade" 
               ipaddr = "10.0.3.4" 
               login = "admin" 
               passwd = "password" 
         } 
}

Example 6-4. Brocade FC-Switch Fencing Devices Named silkworm1 and silkworm2

fence_devices {  
         mdfc1 { 
               agent = "fence_mcdata" 
               ipaddr = "10.0.3.3" 
               login = "admin" 
               passwd = "password" 
         } 
         mdfc2 { 
               agent = "fence_mcdata" 
               ipaddr = "10.0.3.4" 
               login = "admin" 
               passwd = "password" 
         } 
}

Example 6-5. McData FC-Switch Fencing Devices Named mdfc1 and mdfc2

fence_devices {  
         vixel1 { 
               agent = "fence_vixel" 
               ipaddr = "10.0.3.3" 
               passwd = "password" 
         } 
         vixel2 { 
               agent = "fence_vixel" 
               ipaddr = "10.0.3.4" 
               passwd = "password" 
         } 
}

Example 6-6. Vixel FC-Switch Fencing Device Named vixel1 and vixel2

fence_devices {  
         gnbd { 
               agent = "fence_gnbd"
               server = "nodea" 
               server = "nodeb" 
          } 
}

This example shows a fencing device named gnbd with two servers: nodea and nodeb.

Example 6-7. GNBD Fencing Device Named gnbd, without GNBD Multipath

fence_devices {  
         gnbdmp { 
               agent = "fence_gnbd"
               server = "nodea" 
               server = "nodeb" 
               option = "multipath"  <-- Additional entry
               retrys = "5"          <-- Number of retries set to 5
               wait_time = "3"       <-- Wait time between retries set to 3
          } 
}

This example shows a fencing device named gnbdmp with two servers: nodea and nodeb. Because GNBD Multipath is employed, an additional configuration entry under gnbdmp is needed: option = "multipath". Also, for GNBD multipath, the example sets the number of retries to 5 with retrys = 5, and sets the wait time between retries to 3 with wait_time = 3.

Example 6-8. GNBD Fencing Device Named gnbdmp, with GNBD Multipath

fence_devices { 
         riloe1 { 
               agent = "fence_rib" 
               ipaddr = "10.0.4.1" 
               login = "admin" 
               passwd = "password" 
         } 
         riloe2 { 
               agent = "fence_rib" 
               ipaddr = "10.0.4.2" 
               login = "admin" 
               passwd = "password" 
         } 
}

In this example, two RILOE fencing devices are defined for two nodes.

Example 6-9. Two HP-RILOE-Card Fencing Device Named riloe1 and riloe2

fence_devices {  
         xcat { 
               agent = "fence_xcat" 
               rpower = "/opt/xcat/bin/rpower"  
         }  
}

Example 6-10. xCAT Fencing Device Named xcat

fence_devices {  
         egenera { 
               agent = "fence_egenera" 
               cserver = "c-blade1"  
         }  
}

Example 6-11. Egenera BladeFrame Fencing Devices Named and xcat2

fence_devices { 
         admin { 
               agent = "fence_manual" 
         } 
}

Example 6-12. Manual Fencing Device Named admin

WarningWarning
 

Manual fencing should not be used in a production environment. Manual fencing depends on human intervention whenever a node needs recovery. Cluster operation is halted during the intervention.

PrevHomeNext
Creating the cluster.ccs FileUpCreating the nodes.ccs File