Table of contents for Advanced POWER virtualization on IBM eserver p5 servers. Architecture and performance considerations / [Ben Gibbs ... [et al.]].


Bibliographic record and links to related information available from the Library of Congress catalog
Note: Electronic data is machine generated. May be incomplete or contain other coding.


Counter
Chapter 1.  Introduction ................. ................... . . . . . . . . . . 1
1.1  Performance tuning redefined ...................  ............... 3
1.1.1  Understanding performance  ................................ 3
1.1.2  Performance  considerations  ................................ 6
Chapter 2. IBM POWER5 architecture... ........ ........... 9
2.1  Introduction  ........ ....  .  .  .   . .   .  .  .  .  .  .  .. .  . . . . . . . . . . . . . . . . .  10
2.2  Chip design ................................................ 12
2.3 POWER5 enhancements .................................. 13
2.4  POWERS instruction pipelines  .............................. ..14
2.4.1  Instruction fetching  ... . .  . . . . . . . . . . . . . .  . . . . . . .  . . . . . ..........  15
2.4.2  Branch prediction  . . . . . . . . . . . . . . . .  . . .   .  . . .  . .  . .  .  16
2.4.3  Instruction decoding and preprocessing ......... . ........ ... 17
2.4.4  Group  dispatch  . . . . . .  .  .   . .  .   . . . . . . . . . .  .  .  .  .  .  .  .  .   . .  .  17
2.4.5  Register renaming  .  .   .  .  .  . .  .  .  .  .  .   .   . . . . . . . . . . . . . . . . . .  . 18
2.4.6  Instruction execution . .  . . . . . . . . . . . . . . . . . . . . . . .  .  .  .  .  ..........  19
2.5  Caches  .......................  .   .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .   .   .21
2.5.1  Level 2  (L2) cache. .. .. . . . . . . . . . . .  .  . . . .  .  .  .  .  .. .... . . . . . . . . .  25
2.5.2  Level 3  (L3) cache. . . ............... ...................  27
2.5.3  Summary of caches on POWER5................. ......... 30
2.5.4  Address translation resources.............................  30
2.6  Timing  facilities  . . . . . . . . . . . . .  ...................  . .  .....  31
2.7 Dynamic power management ...... . . . . .  . . . . . . . . . . . . . ... . 33
2.8 Processor Utilization Resource Register (PURR)................... 34
2.9  Large  POWERS SMPs  ................... ...... .  .   . . . . . . . . . . . .  36
2.10  Summary ...  . . . . . . . .... ...................  .  . . . . . . . . . . . .  . 40
Chapter 3. Simultaneous multithreading  ...........................  41
3.1  What is multithreading? ................. ................... . . . . . . . 42
3.2 POWER5 simultaneous multithreading features .................... 44
3.2.1  Dynamic switching of thread states. ........... .............. 46
3.2.2  Snooze and snooze delay . . . . . . . . . . . .   . . . . . . . . . . . . . . . .  . 47
3.3  Controlling priority of threads  ......... . ... .   . . . . . . . . . . . . .. . 49
3.3.1 Dynamic resource balancing (DRB) .................. ...... 49
3.3.2  Adjustable thread  priorities  . . . . . . . . . . . . .... . . . . . . . . . . . .  . 50
3.3.3  Thread priority implementation . . . . . . . . . . . . ... . .  . . . . . . . .  . 52
3.4  Software considerations  ............... . . . . . . .... ....  . ..... 55
3.4.1 Simultaneous multithreading aware scheduling ... ............ 55
3.4.2  Thread priorities on AIX  5L V5.3  . . . . . . . . . . . ... . .   . . . . . . ... . 56
3.4.3  Thread priorities on Linux ...... ............  .  . . .  ...... 58
3.4.4  Cache effects  ...................  ...... . ........  . . ... . 58
3.5 Simultaneous multithreading performance ................. ......  59
3.5.1 Engineering and scientific applications ............. ... . .. .. 59
3.5.2  Simultaneous multithreading benchmarks  ....... . . . ....... . 61
3.6  Sum mary ......... . . . . . .  . . . ......... . . . .   .  . . . .  .. . . . .  .  71
Chapter 4. POWER  Hypervisor ............. . .. . ...   .. .. . . ...  73
4.1  POWER Hypervisor implementation  ...... ... . . .  . . . . .  . . . . . . 76
4.1.1  POWER  Hypervisor functions  . . . . . . . . . . . ... . .   . . . . . . . .   79
4.1.2  Micro-Partitioning extensions  ....................... .. . . . . . . . . 85
4.1.3  POWER  Hypervisor design . .  .  ..........  . . . . .. . .. . . . . .  87
4.2 Performance considerations ............ ..   . ..... . . . . . .  . .. 90
Chapter 5. Micro-Partitioning ............. .... . . .. . .... . . . . . ... . 93
5.1 Partitioning on the IBM eServer p5 systems.. .. ... . ....... . .. 94
5.2  Micro-Partitioning implementation ................... ........ .. . 96
5.2.1  Virtual processor dispatching  . . . . .. . . . . . . . . . . . . . . . . .  ... . . 104
5.2.2  Phantom  interrupts  . . . . . . . . . . . . .... .   .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  .  112
5.3 Performance considerations ........ ......................... 115
5.3.1  Micro-Partitioning considerations  .......................... 116
5.3.2  Locking considerations ...... . . . . . . .  . . . . ..... . . ...  .   .  121
5.3.3  Memory affinity considerations . . . . . . . . . . . . . . . . . . . . . ... .  126
5.3.4  Idle partition consideration  ..........   .... . . .  .  .   ... .   127
5.3.5 Application considerations in Micro-Partitioning ... ......... .128
5.3.6  Micro-Partitioning planning guidelines  .. . ............... ...   133
5.4  Summary . . . . . . . . . . .  . . . . ................  .  . . . . . .  .   142
Chapter 6. Virtual I/O  . . . . . . . . . . . .....  ........  . . . . . .  .  .  .  .  .   .  143
6.1  Introduction  . . . . . . . . . . . . ....  .  .  .  .  .  .  .  .  .  . .... .....  .  .  .  144
6.2  POWER Hypervisor support for virtual I/O  ....................... 145
6.2.1  Virtual I/O  infrastructure  . . . . . . . . . . . . . . . . . . . . . . . . . . ... .  147
6.2.2  Types of connections  .  ......... ......  .. .  . . .  . . .  .  149
6.3  The  IBM  Virtual I/O  Server . . . . . . . . . . . . ... . . . . . . . . . . . . . . . . .  152
6.3.1 Providing high availability support ..... ... ...... ....... 156
6.4  Virtual Serial Adapter (VSA) .............................. . . . . . . . . 163
6.5  Virtual Ethernet ....... . . . . . . . . ...................  . .. . . .  . 164
6.5.1  Virtual LAN  ...... . . . . . .... ...................  .  . . . .  . 164
6.5.2  Virtual Ethernet connections . . . . . . . . . . . ..... .   . . . . . . . .  . 169
6.5.3  Benefits of virtual Ethernet . . . . . . . . . . . . .... .   . . . . . . . . .  .  .   170
6.5.4  Limitations and considerations  ........ . . . . .... .   . . . . . . .  . 171
6.5.5 POWER Hypervisor switch implementation ................. 171
6.5.6  Performance considerations . . . . . . . . .... . . . . . . . . . . . . . . . . . . 174
6.5.7 VLAN throughput at different processor entitlements ........... 176
6.5.8 Comparing throughput of VLAN to physical Ethernet ........ . 178
6.5.9  Comparing CPU utilization ............................... 180
6.5.10 Comparing transaction rate and latency .................. 182
6.5.11  VLAN  performance  . . . . . . . . . . . .... . . . . . . . . . . . . . . . . .  .  183
6.5.12 VLAN implementation guidelines ......................... 185
6.6  Shared Ethernet Adapter. ................... ....... . . ........ 186
6.6.1 Shared Ethernet Adapter performance ............... ..... 190
6.6.2  Request/response time and latency ...  ........ .... ......  193
6.7  Implementation guidelines. ............... .  ............... .  196
6.7.1  Guidelines for Shared Ethernet Adapter sizing ................ 197
6.7.2 Guidelines for physical Ethernet sizing ................... . 202
6.7.3 Control of threading in the Shared Ethernet Adapter .......... 204
6.8  Virtual SCSI. ........ . . .  ... .  ............. .. ... . . .  . .  .  .  .  .   205
6.8.1  Client and server interaction . ........ . ........ ........ .  209
6.8.2 AIX 5L V5.3 device configuration for virtual SCSI.............. 210
6.8.3  Interpartition communication . . . . . . . . .  . . . . . . . . . . . .  .  212
6.8.4  Disk considerations . . . . . . . . . . . . . . . ....  ................  215
6.8.5  Configuring for redundancy  . . . . . . . . . .  . . . . . . . . . . . . .  .  217
6.8.6  Performance considerations ............. .  .. . ............  220
6.8.7  Sizing a virtual SCSI server . . . . . ...... ...........  226
6.9  Summary .  . . . . . . . . . . . . . .  .  .....................  . . . . . . . . . . . . .  230
Chapter 7. AIX 5L Version 5.3 operating system support........... 233
7.1  Introduction  ................... ............. . . .... . . . . . . . . .   234
7.1.1  Processors  ................... ......... . . . . . . ....  . . . . .  234
7.1.2  Dynamic re-configuration  .................. . . .. . . ... .  239
7.1.3 Existing performance commands enhancement.............. 239
7.1.4 New performance commands ............................ 248
7.1.5  Paging  space  ............... .  . . . ... .. ...........  ..  251
7.1.6 Logical Volume Manager (LVM) ........ .. . ... . ... . .. . 252
7.1.7  Virtual local area network (VLAN) .... ........ .  . . . . . ..... 254
7.1.8  EtherChannel  ............... .  . .  ... ..............  .. . . .  255
7.1.9  Partition Load Manager .. . . .. .  ........ . .  ... .. .  .  .  .   . . 255
Chapter 8. POWER5 system performance ........................ 257
8.1  Performance commands . . . . . . . . . . . . . . . . . . . .  .  . .. . . . . .  . 258
8.1.1  Iparstat command  . . . . . . . . . .  . . . . . . . . .......  258
8.1.2 mpstat command . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .......... 264
8.1.3  vmstat command . ............ . . . . . . . . . .  . . . . .  .  ...  . 268
8.1.4  iostat command  . . . . . . . . . . . ....  ..  . ........... . . . .  270
8.1.5  sar command  . . . . . . . . . . . .  .  .   . .  . . . . . . . . .  . . .  .... .  .   .  272
8.1.6  topas command. .... . ..... . .. . . . . .  .  .  .  .   . . ... . . .  275
8.1.7  xmperf command  .......  .  .   . . . . . . . . . . . . . . . . .  .  .  .  .   .  278
8.2 Performance tuning approach ................................ 283
8.2.1  Global performance analysis. .. ........... . .  .  ...   .. . .. . 283
8.2.2  CPU  analysis  .  . . . . . . . . . . . . .  ........ . . . . . . . . . .289
8.2.3  Memory analysis . . . . . . . . . . . .   .  .   . . .   .  .  .  .  . .  . .294
8.2.4  Disk I/O  analysis . . . . . . . . . . . . ... . .   . . .  .   ..... .  .  296
8.2.5  Network I/O  analysis . . . . . . . . . . . . . . . .  .  ........ .  .   .  304
Chapter 9. Application tuning  ............. ...  .. .  .  . .......  311
9.1  Performance bottlenecks identification . .  .... . ..... .  . . . .  .  312
9.1.1 Time commands, time utilities, and time routines ....... ...... 314
9.2 Tuning applications using only the compiler . . ....... . ....... . . 317
9.2.1  Compiler brief overview. . .  . . . . . . . . . . ..  .   .  .   .....  317
9.2.2  Most commonly used flags .........  . .  .  .  .  .  .   .  .  321
9.2.3 Compiler directives for performance .. ....... . ..... ..... 327
9.2.4  POWER5 compiler features . .  .  .  .  . ... ......   . . . .... ..  332
9.3  Profiling  applications  . . . . . . . . . .. .  ................. .. .  336
9.3.1 Hardware performance monitor ...... ...... ..... .... ... .336
9.3.2  Profiling  utilities  .............. .. .   .  .  .  .  .  ... ... ...... .343
9.4  Memory management . . . . . . . . . . .   ................. .. .  350
9.5  Optimization of critical sections in the code  ............... .. ...  351
9.5.1 General rules for optimization strategies. .................. . .353
9.5.2  Array optimization . . . . .  .  . . .  . .  . . . . . . . . . . . .  .  353
9.5.3  Loop  optimization  . . . . . . . . . .  ................... . . .  355
9.6  Optimized  libraries .. ................. .... . . . . . . . .... . .  .. . 360
9.6.1 MASS Library ...... ............ ...... .. ........... .. 361
9.6.2  ESSL  library  ......... . . . . . .  ........ . . . ..... . ....... . 368
9.7 Parallel programming general concepts ......................... 370
Chapter 10. Partition Load Manager ...... . ...  .  .  ....  ......  . ..  373
10.1 When and how should I use Partition Load Manager? ......... . 374
10.1.1 Partition Load Manager and other load-balancing tools ........ 374
10.1.2 When to use Partition Load Manager ................... . 376
10.1.3 How to deploy Partition Load Manager. ................ .. 382
10.2 More about Partition Load Manager installation and setup.......... 383
10.2.1 Overview of Partition Load Manager behavior ............... 383
10.2.2 Management versus monitoring modes .................... 385
10.2.3  Configuration file and tunables . . . . . . . . . . . .  .  .  .  .  .  .  .   . .  386
10.3 Managing and monitoring with Partition Load Manager ............ 390
10.3.1  Managing multiple partitions . ....  ... . .. . .  ..  .  .   .......  391
10.3.2 Extra tips about the xlplm command ................... .. 392
10.3.3 Examples of Partition Load Manager commands output ....... 393
10.4 Partition Load Manager performance impact .................... 396
10.4.1 Partition Load Manager resource requirements .............. 396
10.4.2 Partition Load Manager impact on managed partitions ........ 397



Library of Congress subject headings for this publication: Virtual computer systems