IBM RS/6000 Troubleshooting Guide
This guide documents fault diagnosis for the IBM RS/6000 family (machine types 7011 / 7012 / 7013 / 7015 / 7020 / 7025 / 7026 / 7043 / 7044 / 7248). RS/6000 troubleshooting differs significantly from PC troubleshooting in that every RS/6000 has a 3- or 4-digit LED operator panel that displays both BIST (Built-In Self-Test) and POST codes, and that operating-system errors are reported with structured SRN (Service Request Number) codes that map directly to IBM-supplied FRU lists.
The Operator Panel
Every RS/6000 carries an operator panel with a 3-digit or 4-digit LED display (some later 7026 / 7044 systems use a 16-character LCD instead). The display shows:
- Codes during power-on — BIST and POST progress; halt codes if a fault is detected.
- Codes after AIX boots — operating-system service codes; "888" flashing for kernel halt.
- Codes during shutdown — graceful shutdown progress.
The complete reference is the per-machine Operator Guide / Service Guide (SA38-05xx family) plus the kev009 mirror of IBM's "RS/6000 3-Digit Display Codes" document.[1][2]
BIST Codes (Hardware POST)
BIST runs first at power-on, before any firmware initialisation. BIST codes are the lowest-level fault indication.
| Code | Meaning |
|---|---|
| 100 | BIST is running. Cleared on success. |
| 101 | BIST is running. Cleared on success. |
| 102 | BIST checksum on the boot ROM failed. ROM corrupted or failed. |
| 103 | BIST timed out. CPU or service-processor fault. |
| 104 | Equipment Check (general hardware fault). |
| 105 | ROS (Read-Only Storage) test failure. |
| 106 | L2 cache failure (where fitted). |
| 110 | Power Good not received from PSU in time. |
| 111 | Bus interface fault on the planar. |
| 112 | Watchdog timer expired during BIST. |
| 113 | Reset issued by service processor — usually transient. |
| 120–129 | Memory subsystem BIST failures. |
| 130–139 | I/O subsystem BIST failures. |
| 140–149 | SCSI subsystem BIST failures. |
| 150–159 | Graphics adapter BIST failures. |
The exact per-code FRU list is in each machine's service guide. The above is the cross-family pattern.
IPL POST Codes (Firmware Boot)
After BIST, the firmware (proprietary ROS on POWER1/POWER2; PReP on 7248; Open Firmware on CHRP) runs IPL. IPL codes are in the 200–400 range.
| Code | Meaning |
|---|---|
| 200 | Mode switch in SECURE position — boot attempted but blocked.[3] |
| 201 | Checkstop during IPL — major hardware fault, CPU halted.[4] |
| 202 | NVRAM read failure. Replace NVRAM module (see IBM RS/6000 Maintenance Guide). |
| 203 | NVRAM CRC failure. Re-enter SMS configuration. |
| 204 | NVRAM write failure. |
| 205 | Service processor not responding. |
| 210 | Memory configuration error. Re-seat memory cards. |
| 211 | Memory ECC test failure. |
| 220 | I/O slot configuration error. |
| 221 | PCI bus configuration error (CHRP machines). |
| 222 | PCI device enumeration failure. |
| 230 | SCSI controller not responding. |
| 231 | SCSI bus configuration error. |
| 232 | No boot device found on SCSI. |
| 240 | Boot disk not in boot list. |
| 241 | Boot image corrupt. |
| 250 | Network boot started. |
| 260 | Diagnostic mode requested (key in SERVICE position). |
| 299 | IPL completed; AIX kernel handed control. |
The specific code-to-FRU mapping is in the per-machine service guide. The above is the cross-family pattern.
AIX-Generated Codes (After Kernel Boot)
Once AIX is running, the LED panel is driven by the kernel. Codes in the 500–900 range indicate runtime / operational events.
| Code | Meaning |
|---|---|
| 500 | Init started. |
| 511 | Filesystem checks running. |
| 517 | /etc/inittab being processed. |
| 520 | Init complete, daemons starting. |
| 540 | Network initialising. |
| 551 | Login prompt presented. |
| 553 | /etc/rc.tcpip running. |
| 581 | ODM (Object Data Manager) reconfiguring. |
| 700 | Program Interrupt — kernel panic or invalid instruction.[5] |
| 888 | Unexpected system halt (kernel panic / hardware fault). Code flashes. See below.[6] |
The 888 Halt Sequence
When the kernel panics, the operator panel cycles through a sequence beginning with a flashing 888. The format is:
888 102 xxx yyy
Where:
- 102 — software-induced crash (the kernel called the panic routine).
- xxx — crash code:
- 300 = Data Storage Interrupt (DSI; bad memory access).
- 700 = Program Interrupt (invalid instruction, trap, panic).
- 0c0 = successful kernel dump completed.
- 0c5 = kernel dump attempted but failed.
- yyy — dump status code.
Press the Reset button on the operator panel to advance through the sequence. Record every code before resetting.
SRN — Service Request Numbers
When AIX detects a hardware error or when a diagnostic is run, the error is reported as an SRN (Service Request Number) in the format:
sss-rrr
Where sss is the source code (CPU, memory, SCSI, etc.) and rrr is the reason. SRNs map directly to the FRU list in the per-machine service guide. Run diag from AIX to see active and historical SRNs.
Examples (from kev009 LED codes reference):[7]
- 101-xxx — Memory.
- 165-xxx — Planar logic.
- 201-xxx — Memory test.
- 221-xxx — Memory adapter.
- 260-xxx — Display adapter.
- 651-xxx — Internal SCSI device.
- 767-xxx — Graphics adapter.
SCSI-Specific Codes
The 0540-class SCSI error codes appear during IPL or AIX boot when the SCSI subsystem cannot enumerate devices.
| Code area | Meaning | First action |
|---|---|---|
| 0540 | SCSI controller initialisation timeout | Reseat SCSI controller card; check SCSI cable |
| 0552 | SCSI bus parity error | Replace SCSI cable, then terminator, then drive |
| 0556 | SCSI hard reset failure | Check for missing terminator at end of bus |
| 0581 | No bootable device on SCSI bus | Run SMS, verify boot list |
Stiction Diagnosis
If the operator panel halts at a 230 / 232 / 0540 / 0556 SCSI code immediately after power-on and the SCSI drive is silent (not spinning), suspect spindle stiction. See IBM RS/6000 Maintenance Guide for the field stiction fix.
Mode Switch
The operator panel includes a key switch with positions:
- NORMAL — boot the default OS from the boot list.
- SERVICE — boot the standalone diagnostic image (from disk or CD-ROM).
- SECURE — block all boots (used for transport; produces a 200 code on attempted boot).
Always check the mode switch position before troubleshooting an apparent boot failure — a key turned to SECURE is a very common "first day in the lab" symptom.
SMS — System Management Services (CHRP Machines)
On 7025 / 7026 / 7043 / 7044, the SMS menu is the user-facing configuration interface. Entry keys at POST:
- F1 — graphical SMS (at the keyboard icon).
- F4 — text-mode SMS (English).
- F5 — boot from default boot list (5 on an ASCII terminal).
- F6 — boot from the multiboot menu.
- F8 — Open Firmware "ok" prompt.
If the system halts at a POST code before reaching the keyboard icon, SMS cannot be entered — diagnose the halt first.
Memory Faults
- 201 / 211 / 221 — memory test or memory adapter failure. Reseat memory cards; on POWER2 39H/397 the memory must be in matched pairs.
- On 7012-39H / 7012-397 / 7013-590 / 7013-595 — pulling one card of a pair will halt POST with a 210 / 211 code.
- ECC scrubbing on long-uptime systems can produce gradual "memory degraded" entries in errpt — replace the indicated DIMM/SIMM before the second bit error in a word fails outright.
Graphics Faults
- No video, system POSTs to 299 / 551 (login prompt) on serial console — graphics adapter problem. Try a different GXT card.
- Video on initial POST but blank by login prompt — AIX driver loaded for a card that isn't present, or wrong DSP file. Boot to single-user mode and run lsdev -C to verify the graphics device.
- GXT800P / GXT3000P pulling down +5 V — tantalum decoupling cap failed short. Recap.
NVRAM Faults
NVRAM-class faults appear as:
- 202 / 203 / 204 at IPL — NVRAM read / CRC / write failure.
- Boot list reset to defaults every power cycle — Dallas TimeKeeper battery dead.
- Clock at epoch (1 Jan 1970 or 1 Jan 1980) after every cold boot — same.
Fix: replace the Dallas TimeKeeper module (see IBM RS/6000 Maintenance Guide).
Service Processor Faults (Later Machines)
7025 / 7026 / 7044 carry a separate service processor that runs independently of the main CPU. If the service processor fails:
- No LED panel display at power-on — service processor not booting.
- LED display stuck at "STBY" or "0000" — service processor running but not transitioning to IPL.
- Service processor log full — accumulated thermal / fan / ECC events; clear via cfgmgr or the SP menu.
Service processor reset on 7025-F50 / F80 is via the small recessed switch on the rear of the chassis.
PSU Faults
- Dead — no fans, no power: PSU input rectifier or bulk capacitor. Discharge before any work.
- Fans spin briefly, then click-retry: Power Good not asserted. Could be PSU fold-back, shorted planar tantalum, or on 7013 500-series, a fan sense signal missing.[8]
- Boots cold, fails when warm: aged secondary electrolytics.
- Audible whine, smell of fish: RIFA X2 cap venting.
- Rails low/high: PSU feedback path issue. Recap.
Diagnostic Workflow
- Confirm the mode switch is in NORMAL.
- Power on. Watch the LED panel.
- If LED stops at a code, look up the code in the per-machine service guide first; if not found, in the cross-family table above; finally in kev009.[9]
- If LED reaches 299 but AIX does not start, switch to a serial console (DB-9 on the system, 9600/8/N/1) and watch for kernel boot messages.
- If AIX boots but errpt shows ongoing hardware events, run diag to get the SRN.
- Cross-reference the SRN to the FRU list in the service guide; replace the indicated FRU(s).
Common Faults and Resolutions
- No POST, no LED activity — PSU dead. Check rails. Check internal fuse if fitted.
- Halts at 200 — mode switch in SECURE. Turn key.
- Halts at 201 — checkstop. Reseat CPU MCM card; on POWER1/POWER2 7012/7013, suspect SMD electrolyte leakage on planar.
- Halts at 202 / 203 — NVRAM dead. Replace Dallas module.
- Halts at 211 — memory pair mismatch on POWER2. Reseat memory cards.
- Halts at 230 / 232 / 0540 / 0556 — SCSI fault. Cable, terminator, drive stiction.
- Halts at 260 — diagnostic mode (NORMAL key was in SERVICE).
- POST reaches 299 but AIX dies during boot — corrupted AIX rootvg. Boot from AIX install CD and use the maintenance shell.
- Boot loops to SMS — boot list points at a missing device; correct via SMS → Multiboot.
- Random reboots under load — PSU rails sagging, or thermal event due to failed CPU fan.
- Graphics card not detected — Open Firmware did not enumerate it; reseat and check for tantalum failure.
Service Processor Logs (CHRP Machines)
On 7025 / 7026 / 7044, the service processor maintains an event log accessible via:
- The SP menu (boot to SMS, then "Service Processor Setup" → "Read Service Processor Log").
- AIX command snap -a (collects all logs to /tmp/ibmsupt for IBM service).
- AIX command errpt -a (formatted error report).
Reading the 3-digit LED codes
The RS/6000 shows its boot and diagnostic progress, and any fault, as a 3-digit code on the operator-panel LED display. Read the code first — it points straight at the failing area:
| Code | Meaning |
|---|---|
| 215 | Low-voltage condition (irrecoverable) — power-supply fault |
| 221 | NVRAM CRC error — reset by re-running IPL in Service mode |
| 223 | Attempting to IPL from the SCSI devices in the NVRAM boot list (stuck here = boot-device/SCSI problem) |
| 21c | L2 cache not detected |
| F57 | Bad or low backup battery |
| F6A / F7A / F7B | SCSI init / NVRAM init / NVRAM CRC check |
The full published code list is in the references.[10]
NVRAM battery
The RS/6000 keeps its configuration and IPL device list in battery-backed NVRAM. A dead or removed battery loses the settings and can leave the machine unbootable or "confused" — some 43P systems are hard to recover after the backup battery is pulled. Replace a low battery (F57), and after any NVRAM loss re-enter the boot configuration in Service mode.[10]
SCSI and memory
Most RS/6000 boot hangs are the SCSI subsystem (the codes step through SCSI init and the IPL-device search) or bad memory SIMMs. Reseat the SIMMs and the SCSI cabling and terminators, and confirm the boot device against the NVRAM IPL list.[10]
References
- ↑ http://ps-2.kev009.com/jasper/aix/rs-leds.errcodes.html
- ↑ http://ps-2.kev009.com/pimpworks/ibm/aixled.html
- ↑ http://ps-2.kev009.com/jasper/aix/older.rs-leds.errcodes.html
- ↑ http://ps-2.kev009.com/jasper/aix/older.rs-leds.errcodes.html
- ↑ https://sysadminera.com/2017/02/04/the-led-codes-of-aix/
- ↑ https://sysadminera.com/2017/02/04/the-led-codes-of-aix/
- ↑ http://ps-2.kev009.com/jasper/aix/older.rs-leds.errcodes.html
- ↑ https://www.ardent-tool.com/RS6000/docs/pdf/38053100.pdf
- ↑ http://ps-2.kev009.com/jasper/aix/rs-leds.errcodes.html
- ↑ 10.0 10.1 10.2 RS/6000 3-Digit Display Codes, Kev009; and the IBM RS/6000 service manuals. Source for the operator-panel LED codes and the NVRAM-battery behaviour.
Related Pages
References
- RS/6000 3-Digit Display Codes — kev009 mirror. Authoritative LED code reference.
- RS/6000 older 3-Digit Display Codes. Specific BIST/IPL code listings.
- IBM AIX LED diagnostic codes — kev009. Cross-reference list.
- The LED Codes of AIX — sysadminera. Modern restated reference for 888 halt sequence.
- IBM SA38-0531-00 — RS/6000 7013 500-series Installation and Service Guide. PSU + planar service.
- IBM SA38-0512-03 — RS/6000 7043 / 7248 Service Guide. CHRP / PReP service.
- IBM RS/6000 M80 Service Manual. Common firmware error codes chapter (applies broadly across CHRP RS/6000s).
- 7043-140 firmware and SMS keys. F1/F4/F5/F6/F8 entry-key reference.
- IBM TechLib krn4 — RS/6000 boot keys.