To Next Page

To Previous Page

Chapter 3. Central processor complex system design 135

3.9.4 CFCC and Flash Express use

From CFCC Level 19 and later, Flash Express can be used. It improves resilience while

providing cost-effective standby capacity to help manage the potential overflow of IBM

WebSphere MQ shared queues. Structures can now be allocated with a combination of real

memory and SCM that is provided by the Flash Express feature.

For more information about Flash Express and CF Flash use, see Appendix H, “Flash

Express” on page 529.

Default Chapter1
- Front Cover1
- Table of Contents5
Figures17
Notices21
- Trademarks22
IBM Redbooks Promotions23
- Authors25
Preface25
- Comments Welcome27
- Now You Can Become a Published Author, too27
- Stay Connected to IBM Redbooks28
Chapter 1. Introducing IBM Z13S Servers29
- Overview of IBM Z13S Servers30
- Z13S Servers Highlights31
- Z13S Technical Overview39
- Hardware Management Consoles and Support Elements55
- IBM Z Bladecenter Extension (Zbx) Model 00455
  - Blades56
  - IBM Websphere Datapower Integration Appliance XI50 for Zenterprise56
- IBM Z Unified Resource Manager57
- IBM Dynamic Partition Manager57
- Operating Systems and Software58
  - Supported Operating Systems58
  - IBM Compilers61
Chapter 2. Central Processor Complex Hardware Components63
- Frame and Drawers64
  - The Z13S Frame64
- Z13S Frame: Rear and Front View64
  - Pcie I/O Drawer and I/O Drawer Features66
- Z13S (One CPC Drawer) I/O Drawer Configurations66
- Z13S (Two CPC Drawers) I/O Drawer Configurations66
- Processor Drawer Concept67
- Model N10 Components (Top View)68
- Model N20 Components (Top View)69
- CPC Drawer (Front View)69
- CPC Drawer Logical Structure70
  - CPC Drawer Interconnect Topology71
  - Oscillator71
- Drawer to Drawer Communication71
  - System Control72
- Oscillators Cards72
- Conceptual Overview of System Control Elements73
  - CPC Drawer Power74
- Single Chip Modules74
- Redundant Dcas and Blowers for CPC Drawers74
  - Processor Units and System Control Chips75
  - Processor Unit (PU) Chip75
- Single Chip Modules (PU SCM and SC SCM) N20 CPC Drawer75
- PU Chip Floorplan76
  - Processor Unit (Core)77
  - PU Characterization78
- PU Core Floorplan78
  - System Control Chip79
- SC Chip Diagram79
  - Cache Level Structure80
- Memory81
  - Memory Subsystem Topology82
- CPC Drawer Memory Topology82
  - Redundant Array of Independent Memory83
- RAIM Configuration Per Node83
  - Memory Configurations84
- Model N10 Memory Plug Locations84
- Model N20 One Drawer Memory Plug Locations86
- Model N20 Two Drawer Memory Plug Locations88
- Memory Allocation Diagram90
  - Memory Upgrades91
  - Preplanned Memory91
- Reliability, Availability, and Serviceability92
  - RAS in the CPC Memory Subsystem92
  - General Z13S RAS Features93
- Connectivity93
- Model N10 Drawer: Location of the Pcie and IFB Fanouts94
- Model N20 Two CPC Drawer: Locations of the Pcie and IFB Fanouts94
  - Redundant I/O Interconnect95
- Redundant I/O Interconnect for I/O Drawer96
- Model Configurations97
- Redundant I/O Interconnect for Pcie I/O Drawer97
  - Upgrades99
- Z13S Upgrade Paths99
  - Concurrent PU Conversions100
  - Model Capacity Identifier100
  - Model Capacity Identifier and MSU Values101
  - Capacity Backup102
  - On/Off Capacity on Demand and Cps104
- Power and Cooling105
  - Power Considerations105
  - High-Voltage DC Power106
  - Internal Battery Feature106
  - Power Capping106
  - Power Estimation Tool106
  - Cooling Requirements107
- Summary of Z13S Structure107
Chapter 3. Central Processor Complex System Design109
- Overview110
- Design Highlights110
- CPC Drawer Design112
  - Cache Levels and Memory Structure112
- Z13S Cache Levels and Memory Hierarchy112
- Z13S Cache Topology113
- Z13S and Zbc12 Cache Level Comparison114
  - CPC Drawer Interconnect Topology115
- Z13S CPC Drawer Communication Topology115
- Processor Unit Design116
- Point-To-Point Topology Z13S Two CPC Drawers Communication116
  - Simultaneous Multithreading117
  - Single-Instruction Multiple-Data118
- Two Threads Running Simultaneously on the same Processor Core118
- Schematic Representation of Add SIMD Instruction with 16 Elements in each Vector119
- Floating Point Registers Overlaid by Vector Registers119
  - Out-Of-Order Execution120
- Z13S PU Core Logical Diagram121
- Z13S In-Order and Out-Of-Order Core Execution Improvements122
  - Superscalar Processor123
  - Compression and Cryptography Accelerators on a Chip123
  - Decimal Floating Point Accelerator124
- Compression and Cryptography Accelerators on a Core in the Chip124
  - IEEE Floating Point125
  - Processor Error Detection and Recovery125
  - Branch Prediction126
  - Wild Branch126
- PU Error Detection and Recovery126
  - Translation Lookaside Buffer127
  - Instruction Fetching, Decoding, and Grouping127
  - Extended Translation Facility127
  - Instruction Set Extensions128
  - Transactional Execution128
  - Runtime Instrumentation128
- Processor Unit Functions128
  - Overview128
  - Central Processors130
  - Integrated Facility for Linux131
  - Internal Coupling Facility131
  - IBM Z Integrated Information Processor133
- ICF Options: Shared Icfs133
- Logical Flow of Java Code Execution on a Ziip134
  - System Assist Processors137
  - Reserved Processors138
  - Integrated Firmware Processor138
  - Processor Unit Assignment138
  - Sparing Rules139
  - Increased Flexibility with Z/VM Mode Partitions140
- Memory Design140
  - Overview140
  - Main Storage143
  - Expanded Storage143
  - Hardware System Area (HSA)144
- Logical Partitioning144
  - Overview145
  - Storage Operations152
  - Reserved Storage155
  - Logical Partition Storage Granularity156
  - LPAR Dynamic Storage Reconfiguration157
- Intelligent Resource Director157
- IRD LPAR Cluster Example157
- Clustering Technology159
- Sysplex Hardware Overview159
  - Coupling Facility Control Code160
  - Coupling Thin Interrupts161
  - Dynamic CF Dispatching161
- Dynamic CF Dispatching (Shared Cps or Shared ICF Pus)162
  - CFCC and Flash Express Use163
Chapter 4. Central Processor Complex I/O System Structure165
- Introduction to the Infiniband and Pcie for I/O Infrastructure166
  - Infiniband I/O Infrastructure166
  - Pcie I/O Infrastructure166
  - Infiniband Specifications167
  - Pcie Generation 3167
- I/O System Overview168
  - Characteristics168
  - Summary of Supported I/O Features169
- I/O Drawer170
- Pcie I/O Drawer171
- I/O Domains of an I/O Drawer171
- Pcie I/O Drawer172
- Z13S (N20) Connectivity to Pcie I/O Drawers173
- Pcie I/O Drawer and I/O Drawer Offerings174
- Pcie I/O Drawer with 32 Pcie Slots and Four I/O Domains174
- Fanouts175
- Infrastructure for Pcie and Infiniband Coupling Links176
  - Pcie Generation 3 Fanout (FC 0173)177
  - HCA2-C Fanout (FC 0162)177
  - Integrated Coupling Adapter (FC 0172)177
  - HCA3-O (12X IFB) Fanout (FC 0171)178
- OM3 50/125 ΜM Multimode Fiber Cable with MPO Connectors178
  - HCA3-O LR (1X IFB) Fanout (FC 0170)179
  - Fanout Considerations180
- I/O Features (Cards)182
  - I/O Feature Card Ordering Information183
  - Physical Channel Report184
- Connectivity185
  - I/O Feature Support and Configuration Rules186
  - FICON Channels188
  - OSA-Express5S194
  - OSA-Express4S Features196
  - OSA-Express for Ensemble Connectivity199
  - Hipersockets200
- Parallel Sysplex Connectivity202
  - Coupling Links202
- Z13 Parallel Sysplex Coupling Connectivity204
  - Migration Considerations206
- CPC Drawer Front View: Coupling Links207
- HCA3-O Fanouts: Z13S Versus Z114 / Zbc12 Servers208
  - Pulse Per Second Input211
- Cryptographic Functions211
  - CPACF Functions (FC 3863)211
  - Crypto Express5S Feature (FC 0890)211
- Integrated Firmware Processor211
- Flash Express212
  - IBM Flash Express Read/Write Cache213
- 10Gbe Roce Express213
- Pcie I/O Drawer that Is Fully Populated with Flash Express Cards213
- Zedc Express214
Chapter 5. Central Processor Complex Channel Subsystem215
- Channel Subsystem216
- Multiple Channel Subsystems and Multiple Subchannel Sets216
- Output for Display Ios,Config(All) Command221
- Systems CSS: Channel Subsystems with Channel Spanning222
- CSS, LPAR, and Identifier Example223
- I/O Configuration Management224
- Channel Subsystem Summary224
Chapter 6. Cryptography227
- Cryptography in IBM Z13 and Z13S Servers228
- Some Fundamentals on Cryptography228
- Three Levels of Protection with Three Levels of Speed231
- Cryptography on IBM Z13S Servers232
- Cryptographic Hardware Supported on IBM Z13S Servers232
- Z13S Cryptographic Support in Z/Os234
- CP Assist for Cryptographic Functions235
- The Cryptographic Coprocessor CPACF235
  - Cryptographic Synchronous Functions236
  - CPACF Protected Key237
- CPACF Key Wrapping238
- Crypto Express5S239
- Customize Image Profiles: Crypto248
- SE: View LPAR Cryptographic Controls249
- TKE Workstation250
- Cryptographic Functions Comparison253
- Cryptographic Software Support255
Chapter 7. Software Support257
- Operating Systems Summary258
- Support by Operating System258
- Support by Function271
- Cryptographic Support313
- GDPS Virtual Appliance317
- Z/Os Migration Considerations317
- Result of the Display Core Command324
- Simultaneous Multithreading325
- Single-Instruction Multiple-Data325
- The MIDAW Facility326
- IDAW Usage327
  - MIDAW Technical Description327
- MIDAW Format328
- MIDAW Usage328
  - Extended Format Data Sets329
- Iocp330
- Worldwide Port Name Tool331
- IBM Z Bladecenter Extension (Zbx) Model 004 Software Support332
- Software Licensing333
Chapter 8. System Upgrades339
- Upgrade Types340
  - Terminology Related to Cod for Z13S Systems341
  - Permanent Upgrades343
  - Temporary Upgrades344
- Concurrent Upgrades345
  - Customer Initiated Upgrade Facility347
- The Provisioning Architecture348
- Example of Temporary Upgrade Activation Sequence349
  - Summary of Concurrent Upgrade Functions350
- Miscellaneous Equipment Specification Upgrades351
  - MES Upgrade for Processors352
- Memory Sizes and Upgrades for the N10353
  - Preplanned Memory Feature353
- Memory Sizes and Upgrades for the Single Drawer N20354
- Memory Sizes and Upgrades for the Two Drawer N20354
- Feature on Demand Window for Zbx Blades Features Hwms355
  - MES Upgrades for the Zbx355
- Permanent Upgrade through the CIU Facility356
- Permanent Upgrade Order Example357
- CIU-Eligible Order Activation Example358
  - Ordering358
- Machine Profile359
  - Retrieval and Activation359
- IBM Z13S Perform Model Conversion Window360
- Customer Initiated Upgrade Order Activation Number Window360
  - On/Off Capacity on Demand360
  - Overview361
  - Ordering362
- Order On/Off Cod Record Window364
- On/Off Cod Order Example365
  - On/Off Cod Testing365
  - Activation and Deactivation366
- The Capacity Provisioning Infrastructure367
  - IBM Z/Os Capacity Provisioning367
- A Capacity Provisioning Domain369
- Capacity for Planned Event371
  - Capacity Backup373
  - CBU Activation and Deactivation375
- Example of C02 with Three CBU Features376
  - Automatic CBU for Geographically Dispersed Parallel Sysplex377
  - Processors378
- STSI Output on Z13S Server379
- Summary of Capacity On-Demand Offerings381
Chapter 9. Reliability, Availability, and Serviceability383
- The RAS Strategy384
- RAS Functions386
  - Scheduled Outages387
- Enhanced Driver Maintenance388
- RAS Capability for the HMC and SE390
- RAS Capability for Zbx Model 004391
  - Zbx Firmware392
- Considerations for Powerha in Zbx Environment394
- Typical Powerha Cluster Diagram395
- IBM Z Advanced Workload Analysis Reporter395
- Flash Express RAS Components396
- RAS Capability for Flash Express396
Chapter 10. Environmental Requirements397
- IBM Z13S Cabling Options398
- IBM Z13S Power and Cooling398
- Top Exit I/O Cabling Feature399
- Recommended Environmental Conditions402
- Rear Door Heat Exchanger (Left) and Functional Diagram407
  - IBM Zbx Physical Specifications407
- Top Exit Support for the Zbx408
- Energy Management408
  - Power Estimation Tool409
- Maximum Potential Power410
- Power Usage on the System Activity Display410
Chapter 11 . Hardware Management Console and Support Elements416
- Introduction to the HMC and SE416
  - Driver Level 27 HMC and SE Enhancements and Changes417
- Diagnostic Sampling Authorization Control418
- Change LPAR Security419
- Rack-Mounted HMC420
- Ses Location421
  - New Support Elements421
- Configure Backup FTP Server422
- Backup Critical Console Data Destinations422
- Backup Critical Data Destinations of Ses423
- Scheduled Operation for HMC Backup424
- Scheduled Operation for Ses Backup424
  - SE Driver Support with the HMC Driver424
  - HMC Feature Codes425
  - Tree Style User Interface and Classic Style User Interface426
- HMC and SE Connectivity427
- SE Physical Connection427
  - Network Planning for the HMC and SE428
- HMC Connectivity Examples429
  - Hardware Prerequisite Changes429
  - RSF Is Broadband-Only430
- Remote Support Facility431
  - RSF Connections to IBM and Enhanced IBM Service Support System432
- HMC and SE Key Capabilities433
  - Logical Partition Management434
- Change LPAR Group Controls - Group Absolute Capping435
  - Operating System Communication435
- Manage Trusted Signing Certificates436
- Import Remote Certificate Example436
- Configure 3270 Emulators436
  - HMC and SE Microcode437
- Microcode Terms and Interaction438
- System Information: Bundle Level439
  - Monitoring439
- HMC Monitor Task Group440
- Monitors Dashboard Task441
- Display the Activity for an LPAR by Processor Type441
- Display the SMT Usage441
- Monitors Dashboard: Crypto Function Integration442
- Monitors Dashboard - Flash Express Function Integration442
- Environmental Efficiency Statistics443
  - Capacity on Demand Support443
  - Features on Demand Support444
- Customize Console Date and Time446
- Timing Network Window with Scheduled DST and Scheduled Leap Second Offset447
  - NTP Client and Server Support on the HMC448
- HMC NTP Broadband Authentication Support449
- Time Coordination for Zbx Components450
  - Security and User ID Management450
  - System Input/Output Configuration Analyzer on the SE and HMC452
  - Cryptographic Support453
- Cryptographic Configuration Window454
  - Installation Support for Z/VM Using the HMC454
- Enabling Dynamic Partition Manager455
- HMC Welcome Window (CPC in DPM Mode)456
- HMC in an Ensemble456
- Unified Resource Manager Functions and Suites457
  - Ensemble Definition and Management460
  - HMC Availability462
- Ensemble Example with Primary and Alternate Hmcs463
Chapter 12. Performance465
- Z13S to Zbc12, Z114, Z10 BC, and Z9 BC Performance Comparison466
- Performance Information466
- LSPR Workload Suite467
  - Instruction Complexity468
- Memory Hierarchy on the Z13S One CPC Drawer System (Two Nodes)469
- Relative Nest Intensity469
- Relative Nest Intensity470
- The Traditional Factors that Were Used to Categorize Workloads470
- LSPR Workload Categories Based on Relative Nest Intensity471
- New Z/Os Workload Categories Defined472
- Relating Production Workloads to LSPR Workloads472
- Workload Performance Variation473
  - Main Performance Improvement Drivers with Z13S474
Appendix A. IBM Z Appliance Container Infrastructure478
- A.1 What Is Zaci478
- Zaci Framework Basic Outline479
  - Example: Deploying IBM Zaware479
- IBM Zaware Image Profile Based on Zaci480
- Zaci Icon in the HMC Interface480
Appendix B. IBM Z Systems Advanced Workload Analysis Reporter (IBM Zaware) . 453482
- B.1 Troubleshooting in Complex IT Environments482
- B.2 Introducing IBM Zaware483
- IBM Zaware Complements an Existing Environment484
  - B.2.1 Hardware Requirements Overview484
- IBM Zaware Shortens the Business Impact of a Problem485
  - B.2.3 IBM Z/Os Solutions to Improve Problem Diagnostic Procedures485
- B.3 Understanding IBM Zaware Technology486
- Basic Components of the IBM Zaware Environment487
- HMC Image Profile for an IBM Zaware LPAR488
- IBM Zaware Heat Map View Analysis489
- IBM Zaware Bar Score with Intervals490
- Feature on Demand Window for IBM Zaware Feature494
  - B.4.2 IBM Zaware Operating Requirements495
- B.5 Configuring and Using IBM Zaware Virtual Appliance496
Appendix C. Channel Options500
- C.1 Channel Options Supported on Z13S Servers500
- C.2 Maximum Unrepeated Distance for FICON SX Features501
Appendix D. Shared Memory Communications504
- D.1 Shared Memory Communications Overview504
- RDMA Technology Overview505
  - D.2.2 Shared Memory Communications over RDMA505
- Dynamic Transition from TCP to SMC-R506
  - D.2.3 Single Root I/O Virtualization506
- Shared Roce Mode Concepts507
  - D.2.4 Hardware507
- 10Gbe Roce Express508
- 10Gbe Roce Express Sample Configuration509
  - D.2.6 10Gbe Roce Express Configuration Example509
- RNIC and OSD Interaction510
  - D.2.7 Hardware Configuration Definitions511
- Physical Network ID Example512
  - D.2.8 Software Exploitation of SMC-R512
- Reduced Latency and Improved Wall Clock Time with SMC-R513
  - D.2.9 SMC-R Support Overview513
  - D.2.10 SMC-R Use Cases for Z/Os to Z/Os514
- Sysplex Distributor before Roce515
- Sysplex Distributor after Roce516
  - D.2.11 Enabling SMC-R Support in Z/Os Communications Server516
- D.3 Shared Memory Communications - Direct Memory Access517
- Clustered Systems: Multitier Application Solution. RDMA, and DMA518
  - D.3.1 Internal Shared Memory Technology Overview518
- Connecting Two Lpars on the same CPC Using SMC-D518
- Dynamic Transition from TCP to SMC-D by Using Two OSA-Express Adapters519
  - D.3.3 Internal Shared Memory - Introduction520
- Concept of Vpci Adapter Implementation521
- SMC-D Configuration that Uses Ethernet to Provide Connectivity522
- SMC-D Configuration that Uses Hipersockets to Provide Connectivity523
  - D.3.5 Planning Considerations523
- ISM Adapters Shared between Lpars524
  - D.3.7 Sample IOCP FUNCTION Statements524
- Multiple Lpars Connected through Multiple Vlans525
  - D.3.8 Software Exploitation of ISM526
- SMCD Parameter in GLOBALCONFIG527
  - D.3.10 Enabling SMC-D Support in Z/Os Communications Server527
  - D.3.11 SMC-D Support Overview528
- E.1 What Is IBM Dynamic Partition Manager530
Appendix E. IBM Dynamic Partition Manager531
- High-Level View of DPM Implementation531
- E.4 Setting up the DPM Environment532
- Enabling DPM Mode of Operation from the SE CPC Configuration Options532
- Entering the OSA Ports that will be Used by the Management Network533
- DPM Mode Welcome Window534
- Traditional HMC Welcome Window When no Cpcs Are Running in DPM Mode535
- User Options When the HMC Presents the DPM Welcome Window535
  - E.4.1 Defining Partitions in DPM Mode535
- DPM Wizard Welcome Window Options537
  - E.4.2 Summary538
- F.1 Why KVM for IBM Z Systems540
Appendix F. KVM for IBM Z Systems541
- KVM Running in Z Systems Lpars542
  - F.2.1 Storage Connectivity542
  - F.2.3 Hardware Management Console543
- Open Source Virtualization (KVM for IBM Z Systems)544
  - F.2.5 What Comes with KVM for IBM Z Systems544
- F.3 Managing the KVM for IBM Z Systems Environment546
- KVM for IBM Z Systems Management Interface546
- KVM Management by Using the Libvirt API Layers547
  - F.3.1 Hypervisor Performance Manager548
- G.1 Design of Native Pcie I/O Adapter Management550
Appendix G. Native Peripheral Component Interconnect Express551
- G.1.4 Management Tasks552
- I/O Domains and Resource Groups Managed by the IFP551
- G.3 Native Pcie Feature Definitions553
- Sample Output of AO Data or PCHID Report553
- Example of IOCP Statements for Zedc Express and 10Gbe Roce Express555
  - G.3.1 FUNCTION Identifier555
Appendix H. Flash Express558
- H.1 Flash Express Overview558
- Systems Storage Hierarchy558
- Flash Express Pcie Adapter559
- Pcie I/O Drawer that Is Fully Populated with Flash Express Cards559
- H.2 Using Flash Express560
- Sample SE/HMC Window for Flash Express Allocation to LPAR561
- Flash Express Allocation in Z/Os Lpars562
- H.3 Security on Flash Express563
- Integrated Key Controller564
  - H.3.2 Key Serving Topology565
- Key Serving Topology566
  - H.3.3 Error Recovery Scenarios566
Appendix I. GDPS Virtual Appliance570
- GDPS Offerings570
- I.1 GDPS Overview570
- I.2 Overview of GDPS Virtual Appliance572
- Positioning a Virtual Appliance572
- GDPS Virtual Appliance Architecture Overview574
- I.3 GDPS Virtual Appliance Recovery Scenarios575
- GDPS Storage Failover576
  - I.3.2 Unplanned Disk Outage576
  - I.3.3 Disaster Recovery577
Appendix J. IBM Zenterprise Data Compression Express580
- J.1 Overview580
  - Groups581
- J.3 Software Support581
  - J.3.1 Z/VM V6R3 Support with Ptfs582
Related Publications583

Brand	IBM
Model	z13s
Category	Server
Language	English

IBM z13s User Manual

Table of Contents

Questions and Answers:

IBM z13s Specifications

Related product manuals