Hp StorageWorks Scalable File Share Uživatelský manuál Strana 1

HP Scalable File Share User GuideG3.2-0HP Part Number: SFSUGG32-DPublished: October 2009Edition: 4

\ Indicates the continuation of a code example.| Separates items in a list of choices.WARNING A warning calls attention to important information that

HP StorageWorks Scalable File Share Release Notes Version 2.3For documentation of previous versions of HP SFS, see:• HP StorageWorks Scalable File Sha

12

1 What's In This Version1.1 About This ProductHP SFS G3.2-0 uses the Lustre File System on MSA hardware to provide a storage system forstandalone

1 CentOS 5.3 is available for download from the HP Software Depot at:http://www.hp.com/go/softwaredepot1.3.1 Hardware ConfigurationA typical HP SFS sy

Figure 1-1 Platform Overview1.3 Supported Configurations 15

Figure 1-2 Server PairsFigure 1-2 shows typical wiring for server pairs.1.3.1.1 Server Memory RequirementsThe Lustre Operations Manual section 3.1.6 d

IMPORTANT: Memory requirements for HP SFS G3.2-0 have increased from previous versions.Before deciding whether to upgrade to HP SFS G3.2-0, please det

• DL380 G6 server support (required for IB QDR)• The -c option to the gen_hb_config_files.pl script automatically copies the Heartbeatconfiguration fi

2 Installing and Configuring MSA ArraysThis chapter summarizes the installation and configuration steps for MSA2000fc arrays use inHP SFS G3.2-0 syste

© Copyright 2009 Hewlett-Packard Development Company, L.P.Confidential computer software. Valid license from HP required for possession, use or copyin

IMPORTANT: The size of a Lustre MDT or OST is limited to 8 TB. Therefore, any volume createdon the MSA2000 must be less than or equal to 8796 GB. If a

# forallmsas show disks ; doneThe CLI syntax for specifying disks in enclosures differs based on the controller type usedin the array. The following v

correct assignment of multipath priorities. HP recommends mapping all ports to each volumeto facilitate proper hardware failover.a. Create vdisks in t

1. Enable FTP on the MSA with the CLI command:# set protocols ftp enable2. Use FTP from a Linux host to upload log files:# ftp MSAIPaddress3. Log in w

2. If you are running with a firewall, the sendmail firewall port 25 must be open by addingthe following line to /etc/sysconfig/iptables before the fi

3 Installing and Configuring HP SFS Software on ServerNodesThis chapter provides information about installing and configuring HP SFS G3.2-0 software o

3.1 Supported FirmwareFollow the instructions in the documentation which was included with each hardware componentto ensure that you are running the l

3.2 Installation RequirementsA set of HP SFS G3.2-0 file system server nodes should be installed and connected by HP inaccordance with the HP SFS G3.2

The following optional, but recommended, line sets up an Ethernet network interface. More thanone Ethernet interface may be set up using additional ne

During the Kickstart post-installation phase, you are prompted to install the HP SFS G3.2-0 DVDinto the DVD drive:Please insert the HP SFS G3.2-0 DVD

Table of ContentsAbout This Document...9Intended

NOTE: USB drives are not scanned before the installer reads the Kickstart file, so you areprompted with a message indicating that the Kickstart file c

NOTE: The output from Installation Phase 1 is contained in /var/log/postinstall.log.Proceed to “Installation Phase 2”.3.4 Installation Phase 2After th

IMPORTANT: This step must be performed for 10 GigE systems only. Do not use this processon InfiniBand systems.If your system uses Mellanox ConnectX HC

3.5.2 Creating the /etc/hosts fileCreate an /etc/hosts file with the names and IP addresses of all the Ethernet interfaces oneach system in the file s

enabling direct user login access to the file system server nodes. In particular, the shadowpassword information should not be provided through NIS or

This import command should be performed by root on each system that installs signed RPMpackages.3.6 Upgrade InstallationIn some situations you may upg

1. For the first member of the failover pair, stop the Heartbeat service to migrate the Lustrefile system components from this node to its failover pa

6. For the upgrade from SFS G3.0-0 to G3.1-0 or SFS G3.2-0, you must re-create the Heartbeatconfiguration files to account for licensing. For the deta

38

4 Installing and Configuring HP SFS Software on ClientNodesThis chapter provides information about installing and configuring HP SFS G3.2-0 software o

3.5.1 Configuring Ethernet and InfiniBand or 10 GigE Interfaces...323.5.2 Creating the /etc/hosts file.

If the client is using the HP recommended 10 GigE ConnectX cards from Mellanox, the ConnectXEN drivers must be installed. These drivers can be downloa

NOTE: The network addresses shown above are the InfiniBand IPoIB ib0 interfaces for theHP SFS G3.2-0 Management Server (MGS) node, and the MGS failove

5. When successfully completed, the newly built RPMs are available in /usr/src/redhat/RPMS/x86_64. Proceed to “Installation Instructions” (page 40).4.

5 Using HP SFS SoftwareThis chapter provides information about creating, configuring, and using the file system.5.1 Creating a Lustre File SystemThe f

To see the multipath configuration, use the following command. Output will be similar to theexample shown below:# multipath -llmpath7 (3600c0ff000d547

node3,options lnet networks=o2ib0,/dev/mapper/mpath6,/mnt/ost4,ost,testfs,icnode1@o2ib0:icnode2@o2ib0 ,,,,"_netdev,noauto",icnode4@o2ib0

2. Start the file system manually and test for proper operation before configuring Heartbeatto start the file system. Mount the file system components

3. Heartbeat uses one or more of the network interfaces to send Heartbeat messages using IPmulticast. Each failover pair of nodes must have IP multica

You can generate the simple files ha.cf, haresources, and authkeys by hand if necessary.One set of ha.cf with haresources is needed for each failover

The haresources2cib.py script is executed by gen_hb_config_files.pl.5.2.3.2 Editing cib.xmlThe haresources2cib.py script places a number of default va

7 Known Issues and Workarounds...617.1 Server Reboot...

• The .sig and .last files should be removed from /var/lib/heartbeat/crm when anew cib.xml is copied there. Otherwise, Heartbeat ignores the new cib.x

NOTE: Changing the authentication string in /etc/ha.d/authkeys causes Heartbeat toreport numerous warnings instead of error messages.atlas1 heartbeat:

This forces you to manually start the Heartbeat service and the file system after a file systemserver node is rebooted.5.5 Monitoring Failover PairsUs

5.7.1 Examining and TroubleshootingIf your file system is not operating properly, you can refer to information in the Lustre 1.8Operations Manual, PAR

This displays INACTIVE if no recovery is in progress. If any recovery is in progress or complete,the following information appears:status: RECOVERING

The problem is in line #08. The MDT is related to 10.129.10.1@o2ib, but in this example theIP address is for the MGS node not the MDT node. So MDT wil

After a few minutes, the MGS mount is active with df.16. Boot the MDS node.17. Start the Heartbeat service on the MDS node:# service heartbeat startAf

common system performance data such as CPU, disk, and network traffic, it also supportsreporting of both Lustre and InfiniBand statistics. Read/write

58

6 LicensingA valid license is required for normal operation of HP SFS G3.2-0. HP SFS G3.2-0 systems arepreconfigured with the correct license file at

List of Figures1-1 Platform Overview...

[root@atlas1] grep "SFS License" /var/log/messagesFeb 9 17:04:08 atlas1 SfsLicenseAgent: Error: No SFS License file found. Check /var/flexlm

7 Known Issues and WorkaroundsThe following items are known issues and workarounds.7.1 Server RebootAfter the server reboots, it checks the file syste

NOTE: Use the appropriate device in place of /dev/mapper/mpath?b. For example, if the --dryrun command returned:Parameters: mgsnode=172.31.80.1@o2ib m

A HP SFS G3 PerformanceA.1 Benchmark PlatformHP SFS G3, based on Lustre File System Software, is designed to provide the performance andscalability ne

Figure A-2 shows more detail about the storage configuration. The storage comprised a numberof HP MSA2212fc arrays. Each array had a redundant pair of

Figure A-3 Single Stream ThroughputFor a file written on a single OST (a single RAID volume), throughput is in the neighborhood of200 MB/s. As the str

filled with the new data. At the point (14:10:14 in the graph) where the amount of data reachedthe cache limit imposed by Lustre (12 GB), throughput d

Figure A-6 Multi-Client Throughput ScalingIn general, Lustre scales quite well with additional OSS servers if the workload is evenlydistributed over t

A.4 One Shared FileFrequently in HPC clusters, a number of clients share one file either for read or for write. Forexample, each of N clients could wr

Another way to measure throughput is to only average over the time while all the clients areactive. This is represented by the taller, narrower box in

List of Tables1-1 Supported Configurations ...

For workloads that require a lot of disk head movement relative to the amount of data moved,SAS disk drives provide a significant performance benefit.

Each disk shelf in the platform used for deep shelf testing was configured in the same manneras described in “Benchmark Platform” (page 63). The arran

Figure A-12 Client Count Versus Total Throughput (MB/s)A.7.3 Throughput ScalingA single file accessed by eight clients benefits from increased stripin

The OSTs were populated with 146 GB SAS drives. Stripe placement was controlled by defaultoperation of the HP SFS file system software. Specific contr

network buffering parameters were set as described in the documentation for the configurednetwork controller.A.8.2 Single Stream ThroughputThroughput

Figure A-16 Client Count Versus Total Throughput (MB/s)A.8.3 Throughput ScalingAs in “Throughput Scaling” (page 66), a set of 16 clients wrote or read

76

IndexSymbols/etc/hosts fileconfiguring, 3310 GigEclients, 39configuring, 32installation, 31performance, 72Bbenchmark platform, 63Ccache limit, 66cib.x

release notes, 17rolling upgrades, 35Sscaling, 66server security policy, 17shared files, 68stonewalling, 68stonith, 46support, 10Tthroughput scaling,

79

8

About This DocumentThis document provides installation and configuration information for HP Scalable File Share(SFS) G3.2-0. Overviews of installing a