IBM pSeries User Manual

IBM ~ pSeries High Performance Switch Tuning and Debug Guide Version 1.0 April 2005 IBM

pshpstuningguidewp040105.doc Page 10 The overhead in maintaining the file cache can impact the performance of large parallel applications. Much of

pshpstuningguidewp040105.doc Page 11 3.3.1 svmon The svmon command provides information about the virtual memory usage by the kernel and user pro

pshpstuningguidewp040105.doc Page 12 PageSize Inuse Pin Pgsp Virtual 4KB 448221 3687 2675

pshpstuningguidewp040105.doc Page 13 statistics in 5-second intervals, with the first set of statistics being the statistics since the node or LPAR

pshpstuningguidewp040105.doc Page 14 adapter is configured. The volume of reservation is proportional to the number of user windows configured on t

pshpstuningguidewp040105.doc Page 15 3.5 Large pages and IP support One of the most important ways to improve IP performance on the HPS is to ensur

pshpstuningguidewp040105.doc Page 16 If you have eight cards for p690 (or four cards for p655), this command also indicates whether you have full me

pshpstuningguidewp040105.doc Page 17 4.2 LoadLeveler daemons The LoadLeveler® daemons are needed for MPI applications using HPS. However, you can

pshpstuningguidewp040105.doc Page 18 SCHEDD_DEBUG = -D_ALWAYS 4.3 Settings for AIX 5L threads Several variables help you use AIX 5L thr

pshpstuningguidewp040105.doc Page 19 5.0 Debug settings and data collection tools Several debug settings and data collection tools can help you deb

pshpstuningguidewp040105.doc Page 2 Contents 1.0 Introduction...

pshpstuningguidewp040105.doc Page 20 5.3 Affinity LPARs On p690 systems, if you are running with more than one LPAR for each CEC, make sure you are

pshpstuningguidewp040105.doc Page 21 On the HMC GUI, select Service Applications -> Service Focal Point -> Select Serviceable Events. 5.7 err

pshpstuningguidewp040105.doc Page 22 • For HAL libraries: dsh sum /usr/sni/aix52/lib/libhal_r.a • For MPI libraries: dsh sum /usr/lpp/ppe.poe/lib

pshpstuningguidewp040105.doc Page 23 MEMORY_AFFINITY Single Thread Usage(MP_SINGLE_THREAD) Hints Filtered (MP_HINTS_FILTERED) MPI-I/O Buffer Size (M

pshpstuningguidewp040105.doc Page 24 MPCI: sends = 14 MPCI: sendsComplete = 14 MPCI: sendWaitsComplete = 17 MPCI: recvs = 17 MPCI: recvWai

pshpstuningguidewp040105.doc Page 25 Run the following command: /usr/sbin/ifsn_dump -a The data is collected in sni.snap (sni_dump.out.Z), and pro

pshpstuningguidewp040105.doc Page 26 To help you isolate the exact cause of packet drops, the ifsn_dump -a command also lists the following debug st

pshpstuningguidewp040105.doc Page 27 There are two routes. sending packet using route No. 1 ml ip address structure, starting: ml flag (ml

pshpstuningguidewp040105.doc Page 28 MAC WOF (2F870): Bit: 1 [. . .] 5.12.4 Packets dropped in the switch hardware If a pack

pshpstuningguidewp040105.doc Page 29 5.14 LAPI_DEBUG_COMM_TIMEOUT If the LAPI protocol experiences communication timeouts, set the environment vari

pshpstuningguidewp040105.doc Page 3 5.10 MP_PRINTENV ... 22 5.11

pshpstuningguidewp040105.doc Page 30 5.16 AIX 5L trace for daemon activity If you suspect that a system daemon is causing a performance problem on

pshpstuningguidewp040105.doc Page 31 7.2 MPI documentation Parallel Environment for AIX 5L V4.1.1 Hitchhiker's Guide, SA22-7947-01 Parallel E

pshpstuningguidewp040105.doc Page 32 © IBM Corporation 2005 IBM Corporation Marketing Communications

pshpstuningguidewp040105.doc Page 4 1.0 Introduction This paper is intended to help you tune and debug the performance of the IBM ® pSeries® High P

pshpstuningguidewp040105.doc Page 5 2.0 Tunables and settings for switch software To optimize the HPS, you can set shell variables for Parallel Env

pshpstuningguidewp040105.doc Page 6 thread, and from within the MPI/LAPI polling code that is invoked when the application makes blocking MPI calls.

pshpstuningguidewp040105.doc Page 7 2.1.5 MP_TASK_AFFINITY Setting MP_TASK_AFFINITY to SNI tells parallel operating environment (POE) to bind each

pshpstuningguidewp040105.doc Page 8 Sometimes MPI-IO is used in an application as if it were basic POSIX read/write, either because there is no need

pshpstuningguidewp040105.doc Page 9 rfifosize 0x1000000 receive fifo size False rpoolsize 0x02000000 IP receiv